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CHAPTER  1 


Executive  Summary 

Microwave  receivers  play  a  vital  role  in  Electronic  Warfare  (EW)  environments  for  passive  identification  and 
localization  of  unknown  targets  emitting  high-frequency  electro-magnetic  signals.  These  Receivers  process  signals 
received  by  Microwave  band  radars  and  majority  of  these  receivers  utilize  analog  signal  processing  tools  and 
techniques.  Microwave  signals  have  very  high  frequency  content  and  have  wide  bandwidths.  As  of  now,  there  are 
no  EW  receivers  that  process  microwave  radar  signals  entirely  in  the  digital  domain.  It  is  expected  that,  with 
the  emergence  of  increasingly  faster  and  inexpensive  digital  computers  and  high-speed  A/D  converters,  digital 
processing  of  microwave  signals  would  most  certainly  be  the  way  of  the  future.  One  of  the  main  purpose  of  this 
project  had  been  to  complement  the  research  on  Digital  Microwave  Receiver  Design  being  conducted  at  the  EW 
Laboratory  at  WPAFB,  Dayton,  Ohio. 

In  addition  to  the  digital  receiver  design  problem,  some  fundamental  theoretical  aspects  of  several  classical 
System  Identification  problems  as  well  as  high-speed  implementation  of  various  Signal  Processing  algorithms 
have  also  been  addressed  as  part  of  this  project.  In  particular,  a  unified  framework  has  been  developed  for 
optimal  estimation  of  rational  transfer  function  parameters  from  prescribed  Time-Domain  or  Frequency- Domain 
specifications.  This  powerful  unifying  theoretical  framework  for  System  Identification  appear  to  have  remained 
mostly  unrecognized  and  un-utilized.  Apart  from  the  digital  EW  receiver  design  problem,  the  proposed  theoretical 
foundation  is  expected  to  have  a  broad  range  of  applications  in  rational  modeling. 

High-speed  implementation  of  digital  signal  processing  algorithms  on  Multiprocessor  Architecture  is  an  im¬ 
portant  topic  of  current  interest  in  recent  Signal  Processing  literature.  In  EW  applications  as  well  as  in  other 
hardware  implementations  of  digital  systems,  high-speed  architecture  would  no  doubt  play  an  important  role. 
In  view  of  this,  high-speed  hardware  implementation  of  a  few  important  signal  processing  algorithms  have  been 
addressed  as  the  final  part  of  this  project. 

As  noted  above,  significant  progress  has  been  made  during  the  course  of  this  research  project.  Several 
problems  of  current  interest  have  been  addressed  and  solved  satisfactorily.  Most  of  the  new  results  were  proposed  in 
the  original  proposal  although  some  intermediate  work  had  been  undertaken  as  the  needs  arose  at  the  Wright  Labs. 
This  Final  Technical  Report  contains  the  details  of  all  the  results  of  the  research  that  have  been  accomplished 
over  the  entire  period  covered  by  the  project.  It  may  be  noted  that  some  of  the  results  presented  in  this  report 
were  initiated  as  part  of  the  work  on  the  original  proposal  (Grant  No.  AFOSR-F49620-93-1-0014). 

The  importance  of  any  research  may  perhaps  be  best  judged  by  the  quality  of  publications  it  gen¬ 
erates.  Consequently,  a  significant  amount  of  time  has  been  devoted  on  preparing  Journal  and  Confer¬ 
ence  articles  in  order  to  report  the  findings  of  this  project.  Most  of  the  results  contained  in  this  report 
have  been  published/accepted/presented  in  internationally  recognized  and  top  quality  Signal  Processing  Jour¬ 
nals/Conferences  although  some  recent  results  are  currently  under  review/preparation  for  future  publication. 
The  papers/publications  ensuing  from  this  research  are  listed  at  the  end  of  this  introductory  Chapter.  Copies  of 
the  papers  and  publications  can  be  made  available  to  the  Program  Monitor,  if  desired. 

The  research  conducted  under  this  project  can  be  categorized  primarily  into  two  broad  themes,  viz., 

(i)  Digital  EW  Receiver  Design  Problems  :  The  problems  addressed  are  as  follows  : 

(a)  A  high-resolution  method  for  AOA  estimation  using  Minimum-Norm  Method  that  does  not  rely  on  any 
Eigendecomposition 
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(b)  Statistical  Perturbation  Analysis  of  the  DFT-based  Minimum-Norm  Method  proposed  in  part-a. 

(c)  A  high-resolution  Maximum-Likelihood  method  for  frequency  estimation  that  guarantees  unit-circle 
roots 

(d)  Two  methods  for  superior  estimation  of  AR  and  ARMA  parameters  when  the  observation  data  is  noisy 

(e)  Time-Domain  algorithms  for  detection  of  Electronic  Warfare  Signals  in  the  presence  of  Noise 

(f)  Pipelined-Adaptive  Tracking  of  Multiple  Sinusoidal  Frequencies. 

(ii)  System  Identification  and  Hardware  Implementation  Problems  : 

(a)  Optimal  identification  of  1-D  Rational  Systems  from  Input-Output  Data 

(b)  Optimal  identification  of  1-D  Rational  Systems  in  the  Frequency  Domain 

(c)  Optimal  Identification  of  All-Pole  Rational  Systems  in  Time-Domain 

(d)  Design  of  Denominator  Separable  2-D  IIR  Filters  from  Spatial-Impulse  Response  Data 

(e)  Design  of  Denominator  Separable  2-D  IIR  Filters  from  2-D  Frequency  Response  Data 

(f)  Design  of  2-D  IIR  Filters  with  non-separable  denominator  from  Spatial-Impulse  Response  Data 

(g)  High-speed  pipelined  implementation  of  1-D  Recursive  Filters  based  on  a  new  Distributed  Look-Ahead 
scheme. 

(h)  Optimal  Estimation  of  LA  filters. 

(i)  High-speed  pipelined  implementation  of  2-D  Recursive  Filters  based  on  the  Distributed  Look-Ahead 
scheme  proposed  in  part-g. 

The  report  is  organized  as  follows:  In  Chapter  2,  the  research  results  on  Digital  EW  receiver  design  related 
problems  are  reported  whereas  in  Chapter  3,  the  System  Identification  and  Hardware  Implementation  areas  are 
covered  with  complete  details.  Individual  Chapters  are  divided  into  several  Sections  by  topics.  In  the  following 
paragraphs  the  main  results  obtained  in  these  each  of  these  sections  are  summarized  briefly. 

CHAPTER  2.  The  Digital  Microwave  Receiver  Design  Problem 

Section  -  2.1  :  Superresolution  without  Eigendecomposition  :  Method  and  Perturbation  Analysis  : 
Many  existing  high-resolution  methods,  such  as  MUSIC  or  Minimum-Norm  Method,  rely  on  special-purpose 
hardware  or  software  for  obtaining  the  signal  and  noise  subspace  eigenvectors  of  Autocorrelation  (AC)  matrices. 
In  this  project,  we  have  developed  a  new  DFT-based  high-resolution  frequency  estimation  algorithm  which  does 
not  require  any  eigendecomposition  and  hence,  it  is  much  less  computation  intensive.  It  has  been  demonstrated 
that  the  DFT  of  the  AC  matrix  (DFT-of-AC)  essentially  performs  an  equivalent  task  of  separating  the  signal 
and  noise  subspaces.  Furthermore,  when  the  signal-subspace  part  of  the  DFT-of-AC  vectors  are  used  in  MNM, 
almost  identical  high-resolution  AO  A  estimates  are  produced.  The  results  have  been  published  as  a  Journal  paper 
[110]  and  has  been  presented  at  ICASSP-94  [119].  It  may  be  noted  here  that  according  to  one  of  the  anonymous 
reviewers  of  the  journal  paper,  this  work  is  a  “significant  breakthrough  in  source  localization”. 

In  the  later  part  of  this  Section,  we  present  a  detailed  theoretical  Perturbation  Analysis  of  the  estimates 
produced  by  the  D-MNM  algorithm.  The  theoretical  results  closely  corroborate  and  confirm  the  superior  perfor¬ 
mance  observed  in  simulations.  The  results  indicate  that  the  high-resolution  performance  of  D-MNM  is  uniformly 
superior  than  its  eigen-based  counterpart,  especially  at  low  SNR.  The  performance  is  also  superior  than  the  eigen- 
based  root-MUSIC  method  at  low  SNR.  Furthermore,  D-MNM  appears  to  provide  better  success  rate  among  all 
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methods  at  low  SNR.  Close  match  between  the  theoretical  and  simulated  performance  verifies  the  validity  of  the 
formulae  derived  here.  Preliminary  work  has  been  presented  at  ASILOMAR-94  [118]  and  a  detailed  version  is 
under  preparation  for  a  Journal  paper  [125]. 

Section  -  2.2  :  Maximum- Likelihood  Method  with  Exact  Constraints  :  A  recently  proposed  approximate 
Maximum- Likelihood  Estimator  (MLE)  of  multiple  exponentials,  converts  the  frequency  estimation  problem  into 
a  problem  of  estimating  the  coefficients  of  a  ^-polynomial  with  roots  at  the  desired  frequencies.  Theoretically, 
the  roots  of  the  estimated  polynomial  should  fall  on  the  unit  circle.  But  MLE,  as  originally  proposed,  does  not 
guarantee  unit  circle  roots.  This  drawback  sometimes  causes  merged  frequency  estimates,  especially  at  low  SNR. 
If  all  the  sufficient  conditions  for  the  z-polynomial  to  have  unit  circle  roots  are  incorporated,  the  optimization 
problem  becomes  too  nonlinear  and  it  loses  the  desirable  weighted-quadratic  structure  of  MLE.  In  this  work,  the 
exact  constraints  are  imposed  on  each  of  the  lst-order  factors  corresponding  to  individual  frequencies  for  ensuring 
unit  circle  roots.  The  constraints  are  applied  during  optimization  alternately  for  each  frequency.  In  the  absence  of 
any  merged  frequency  estimates,  the  RMS  values  more  closely  approach  the  theoretical  Cramer- Rao  (CR)  bound 
at  low  SNR  levels.  The  work  has  been  published  as  a  Journal  paper  [15]. 

Section  -  2.3  :  Improved  AR-Parameter  Estimation  From  Noisy  Observation  Data  :  Auto-Regressive 
(AR)  modeling  is  the  most  widely  used  approach  for  model-based  spectrum  estimation.  But  almost  all  the 
existing  methods  for  AR-parameter  estimation  show  severe  degradation  if  the  observed  signal  is  corrupted  with 
noise.  In  fact,  all  the  commonly  used  techniques,  such  as,  Autocorrelation  Method  (AM),  Covariance  Method 
(CM),  Modified  Covariance  Method  and  their  variations,  give  poor  Power  Spectral  Density  (PSD)  estimates 
when  the  observations  are  noisy.  In  this  part  of  the  project,  a  data- adaptive  pre-filtering  approach  is  presented 
to  address  this  problem.  The  results  indicate  that  when  only  noisy  data  is  available  for  modeling,  the  proposed 
technique  gives  more  accurate  PSD  estimates  than  the  commonly  used  methods.  A  conference  paper  on  this  work 
have  been  accepted  [121]  and  a  more  comprehensive  version  is  under  preparation  for  publication  as  a  Journal 
paper. 

Section  -  2.4  :  Improved  ARMA-Parameter  Estimation  From  Noisy  Observation  Data  :  Existing 
methods  for  ARMA  modeling  assume  that  the  available  process  is  produced  by  an  ARMA  system  driven  by  a 
white  input  process,  i.e.y  the  observed  process  is  considered  to  be  pure  ARMA.  In  practice,  the  available  data 
usually  have  observation  noise  added  to  it  but  the  ARMA  methods  do  not  address  this  problem.  Simulations 
show  that  performance  of  the  existing  ARMA  methods  deteriorate  when  the  observation  process  is  noisy.  In  this 
part  of  the  project  a  new  ARMA  algorithm  is  given  which  utilizes  a  recently  developed  deterministic  rational 
system  identification  method  (OM-IO)  that  minimizes  the  modeling  or  output  error  norm.  The  algorithm  first 
estimates  the  input  process  and  then  invokes  OM-IO  using  the  input-output  data.  Simulations  indicate  that  the 
proposed  method  is  quite  effective  even  at  low  SNR  observation  data.  A  conference  paper  on  this  work  has  been 
accepted  [120]  and  a  more  comprehensive  version  is  under  preparation  for  publication  as  a  Journal  paper. 

Section  -  2.5  :  Time-Domain  Detection  of  Electronic  Warfare  Signals  in  Noise  :  Almost  all  existing 
AOA/RF  estimation  algorithms  assume  that  the  signal  is  already  present  in  the  observed  data.  But  in  the  passive 
mode  of  operations  of  EW  applications,  source  signals  may  not  be  present  at  all  within  the  observation  window, 
or  the  signals  may  fill  only  a  part  of  the  estimation  window.  In  either  case,  any  frequency  estimation  algorithm 
would  essentially  produce  erroneous  or  noise  frequencies  because  the  observed  signal  would  not  satisfy  the  model 
assumed  by  the  estimation  algorithm.  Considering  the  relatively  high  computational  burden,  any  estimation 
method  should  be  invoked  only  when  a  detection  scheme  indicates  high  probability  of  presence  of  threat.  In 
this  part  of  the  project,  the  theory  of  detecting  sinusoids  from  Quantized  and  Noisy  time-domain  observation 
samples  have  been  developed.  The  theoretical  work  on  single/multiple  samples  is  mostly  complete.  Studies  with 
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Quantized  data  have  also  been  performed  and  the  results  appear  reasonably  good.  Lab  tests  for  the  Envelope 
Detection  and  Square-Law  cases  have  been  conducted  at  Wright  Labs  with  satisfactory  results. 

Section  -  2.6  :  Pipelined- Adaptive  Tracking  of  Multiple  Sinusoidal  Frequencies  :  New  Pipelined- 
Adaptive  algorithms  are  proposed  for  tracking  multiple  Frequencies  or  Angles-of- Arrival  (AO A)  of  moving  targets. 
Pipelining  of  adaptive  filters  pose  a  critical  challenge  because  of  the  timing  mismatch  arising  from  the  feedback 
signals.  In  this  work,  some  relaxation  techniques  have  been  utilized  to  pipeline  adaptive  algorithms  for  high¬ 
speed  tracking  of  frequency/AOAs.  Two  adaptive  tracking  algorithms  have  been  mapped  into  pipelined  forms, 
namely  Least-Mean  Squares  (LMS)  and  Recursive  Least-Squares  (RLS).  Preliminary  results  have  been  presented 
at  ISCAS-96  [115]  and  a  Journal  version  is  under  preparation  for  possible  publication  [129]. 

CHAPTER  3.  System  Identification  and  Hardware  Implementation  Problems 

Fundamental  contributions  have  been  made  in  1-D  and  2-D  Rational  System  Identification  theory.  Several  key 
journal  papers  have  been  published/accepted  and  a  number  of  conference  publications  have  also  been  generated. 
The  proposed  comprehensive  framework  encompasses  a  large  class  of  Identification  problems  including,  (a)  Input- 
Output  data  [18,  124],  (b)  Impulse  Response  Data  :  AR  case  [19,  123],  ARMA  case  [Ill]  and  (c)  Frequency 
Response  Data  [13,  122],  (d)  Multivariable  System  Identification  [17]  and  also  for  shaping  Time  responses  of 
Minimum  Phase  Systems  [14].  Key  results  are  summarized  below. 

Section  -  3.1  :  Identification  of  1-D  Rational  Systems  from  Input-Output  Data  :  A  theoretical  and 
algorithmic  framework  is  proposed  for  optimal  identification  of  rational  transfer  function  parameters  of  discrete¬ 
time  linear  systems  from  Input-Output  (10)  data.  The  nonlinear  criterion  is  theoretically  decoupled  into  a  purely 
linear  problem  for  estimating  the  optimal  numerator  and  a  nonlinear  problem  for  the  optimal  denominator.  The 
proposed  decoupled  approach  has  reduced  computational  requirements  when  compared  to  existing  algorithms 
which  estimate  the  parameters  simultaneously.  This  research  has  led  to  one  Journal  paper  [18]  and  a  Conference 
paper  [124]. 

Section  -  3.2  :  Identification  of  1-D  Rational  Systems  in  the  Frequency  Domain  :  A  new  Frequency- 
Domain  (FD)  approach  has  been  developed  for  optimal  estimation  of  rational  transfer  functions  coefficients.  The 
proposed  method  seeks  to  match  any  arbitrarily-shaped  FD  specifications  in  the  Least-Squares  (LS)  sense.  The 
desired  specifications  may  be  arbitrarily  spaced  in  frequency.  The  design  is  performed  directly  in  the  digital  domain 
and  no  analog  to  digital  transformation  is  necessary.  The  proposed  method  makes  use  of  the  inherent  mathematical 
structure  in  this  rational  modeling  problem  to  theoretically  decouple  the  numerator  and  denominator  estimation 
problems  into  two  smaller  dimensional  problems.  The  denominator  criterion  is  nonlinear  but  possesses  a  weighted- 
quadratic  structure  which  is  convenient  for  iterative  optimization.  The  optimal  numerator  is  found  linearly  by 
solving  a  set  of  simultaneous  equations.  The  decoupled  criteria  retain  the  global  optimality  properties.  The 
performance  of  the  algorithm  is  demonstrated  with  some  simulation  examples.  This  research  has  led  to  one 
Journal  paper  [13]  and  a  Conference  paper  [122]. 

Section  -  3.3  :  Identification  of  All-Pole  Rational  Systems  in  Time-Domain  :  An  algorithm  is  proposed 
for  optimal  estimation  of  the  parameters  of  Auto-Regressive  (AR)  or  all-pole  transfer  function  models  from 
prescribed  impulse  response  data.  The  transfer  function  coefficients  are  estimated  by  minimizing  the  ^2-norm  of 
the  exact  model  fitting  error.  Existing  methods  either  minimize  equation  errors  or  modify  the  true  non-linear 
fitting  error  criterion.  In  the  proposed  method,  the  multidimensional  nonlinear  error  criterion  has  been  decoupled 
into  a  purely  linear  and  a  nonlinear  subproblem.  Global  optimality  properties  of  the  decoupled  estimators 
have  been  established.  For  data  corrupted  with  Gaussianly  distributed  noise,  the  proposed  method  produces 
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Maximum-Likelihood  Estimates  (MLE)  of  the  AR-parameters.  The  inherent  mathematical  structure  in  the  non¬ 
linear  subproblem  is  exploited  in  formulating  an  efficient  iterative  computational  algorithm  for  its  minimization. 
The  proposed  algorithm  provides  an  useful  computational  tool  based  on  appropriate  theoretical  foundation  for 
accurate  modeling  of  all-pole  systems  from  prescribed  impulse  response  data.  The  effectiveness  of  the  algorithm 
has  been  demonstrated  with  several  simulation  examples.  This  research  has  led  to  one  Journal  paper  [19]  and  a 
Conference  paper  [123]. 

Section  -  3.4  :  Design  of  Denominator  Separable  2-D  HR  Filters  :  This  work  extends  the  1-D  results 
in  [Ill]  to  2-D  system  identification.  In  this  part  of  the  report,  the  optimal  design  of  an  important  class  of 
two-dimensional  (2-D)  digital  HR  filters  from  spatial  impulse  response  data  is  addressed.  The  denominator  of 
the  desired  2-D  filter  is  assumed  to  be  separable  into  two  1-D  factors.  The  filter  coefficients  are  estimated 
by  minimizing  the  f2-norm  of  the  error  between  the  prescribed  and  the  estimated  spatial  domain  responses. 
The  denominator  and  numerator  estimation  problems  are  theoretically  decoupled  into  separate  problems.  The 
decoupled  criteria  have  reduced  dimensionality.  The  denominator  criterion  is  simultaneously  optimized  w.r.t.  the 
coefficients  in  both  dimensions  using  an  iterative  algorithm.  The  numerator  coefficients  are  found  in  a  straight¬ 
forward  manner.  If  the  desired  response  is  known  to  be  symmetric,  the  proposed  algorithm  can  be  constrained 
to  have  separable-denominators.  Initial  results  have  been  published  as  a  Journal  paper  [16]  and  some  further 
developments  are  currently  being  considered  [12]. 

Section  -  3.5  :  Optimal  Frequency  Domain  Design  of  Denominator  Separable  Two-Dimensional 
Digital  HR  Filters  :  Classical  design  techniques  using  Butterworth,  Chebyshev  or  Elliptic  polynomial  are  only 
limited  particular  types  of  design  specifications,  such  as  Bandpass,  lowpass  etc.  A  least-squares  technique  is 
presented  for  designing  quarter-plane  separable-denominator  2-D  IIR  filters  to  best  approximate  prescribed  fre¬ 
quency  domain  (FD)  specification  of  any  arbitrary  shape.  Structured  Matrix  Approximation  approach  is  utilized 
to  show  that  the  FD  error  vector  is  linearly  related  to  the  2-D  numerator  coefficients  whereas  the  relationship 
with  the  2-D  denominators  is  quasi-linear.  Furthermore,  the  numerator  and  denominator  estimation  problems  are 
theoretically  decoupled.  The  quasi-linear  relationship  with  the  denominator  is  used  to  formulate  an  algorithm  for 
iterative  estimation  of  the  denominator.  The  numerator  is  found  in  one  step  using  the  estimated  denominator. 
Computer  simulations  show  the  effectiveness  of  the  proposed  method  and  its  superior  performance  compared  to 
several  existing  methods.  This  work  has  been  presented  at  ICASSP-95  [117].  A  detailed  version  is  also  under 
preparation  for  a  Journal  paper  [130]. 

Section  -  3.6  :  Optimal  Spatial-Domain  Design  of  2-D  IIR  Filters  :  In  this  Section  we  present  a  structured 
matrix  approximation  framework  to  develop  the  most  general  form  for  optimal  least-squares  (LS)  design  of  2-D 
recursive  filters  from  prescribed  spatial  domain  data.  Unlike  the  work  in  Section  3.4,  no  separability  is  assumed 
for  the  2D  denominator.  Utilizing  matrix  structures  inherent  in  this  problem  it  is  shown  that  the  exact  t2  error 
has  a  purely  linear  relationship  with  the  2-D  numerator  parameters  whereas  the  2-D  denominator  coefficients  are 
nonlinearly  related  to  the  error.  But  more  interestingly,  the  denominator  and  numerator  estimation  problems 
are  theoretically  decoupled  into  separate  problems  without  affecting  any  optimality  properties.  In  the  decoupled 
form,  the  numerator  estimation  problem  is  shown  to  be  purely  linear.  For  estimating  the  denominator  also,  it  is 
shown  that  the  decoupled  i2  error  vector  possesses  a  quasi-linear  relationship  with  the  denominator  coefficients. 
Decoupled  estimation  leads  to  reduced  computational  complexity  because  there  is  no  need  for  iterating  on  the 
numerators.  Simulation  results  indicate  that  for  several  common  filer  design  problems,  the  proposed  general 
version  performs  better  than  the  separable  design  developed  earlier  in  Section  3.4.  Preliminary  results  have  been 
presented  at  ISCAS-95  [115]  and  a  Journal  version  is  under  consideration  for  possible  publication  [II]. 

Section  -  3.7  :  Distributed  Look-Ahead  :  A  General  Approach  for  Pipelining  Recursive  Digital  Fil- 
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ters  :  A  new  Look-Ahead  (LA)  scheme,  Distributed  Look-Ahead  (DLA),  is  proposed  for  pipelined  implementation 
of  recursive  digital  filters.  It  is  established  that  in  case  of  many  recursive  filters,  DLA  can  provide  equivalent  and 
stable  implementation  with  reduced  pipeline  delay  and  hardware  complexity,  when  compared  with  some  existing 
LA  schemes.  The  existing  Scattered  Look-ahead  implementation  achieves  stability  at  the  cost  of  increased  multi¬ 
plication  and  latch  complexities  and  considerable  delay  in  output  generation.  The  Clustered  look-ahead  approach 
can  not  always  guarantee  stability.  This  work  shows  that,  in  order  to  attain  stability,  the  output  samples  need 
not  be  clustered  or  equally  scattered.  Indeed,  in  many  filter  design  problems,  stability  can  be  maintained  by  using 
unequally  distributed  past  output  samples.  When  compared  with  the  scattered  approach,  the  proposed  scheme 
uses  fewer  number  of  pole-zero  cancelations  and  the  introduced  roots  are  not  necessarily  at  the  same  radii  as 
the  original  filter  poles/  Hence,  the  proposed  DLA  scheme  has  reduced  multiplication  and  latch  complexities, 
higher  area-efficiency  and  it  produces  outputs  with  reduced  delays.  Preliminary  results  have  been  presented  at 
ICASSP-96  [113]  and  ISCAS-96  [114]  and  a  Journal  version  is  under  preparation  for  possible  publication  [126]. 

Section  -  3.8  :  Optimal  Least-Squares  Design  of  Pipelined  Recursive  Filters  in  the  Time-Domain 
:  Currently,  look-ahead  (LA)  pipelined  recursive  filters  are  obtained  primarily  via  transformation  of  a  given 
un-pipelined  transfer  function.  For  these  approaches,  it  is  assumed  that  the  un-pipelined  transfer  function  has 
already  been  designed  as  an  intermediate  step.  In  this  Section,  we  present  a  new  algorithm  (OM-LA)  for  direct 
and  optimal  estimation  of  the  coefficients  of  recursive  filters  in  look-ahead  pipelined  form.  OM-LA  is  developed  by 
appropriate  modification  of  a  recently  proposed  optimal  method  (OM)  for  designing  un-pipelined  filters  (developed 
previously  by  the  PI  as  part  of  a  project  supported  by  the  AFOSR).  It  is  demonstrated  that  the  proposed  one-step 
approximation  can  achieve  superior  match  with  reduced  pipelined  filter  order  because  it  does  not  rely  on  pole-zero 
cancelations  as  in  current  LA  pipelining  approaches.  It  is  also  shown  that  the  denominator  polynomial  can  be 
constrained  to  possess  any  of  the  possible  look-ahead  configurations.  Unlike  some  existing  methods,  OM-LA 
minimizes  the  true  time-domain  fitting  error-norm  between  the  prescribed  and  the  estimated  impulse  response 
and  produces  superior  results.  Preliminary  results  have  been  presented  at  ICASSP-96  [112]  and  a  Journal  version 
is  under  preparation  for  possible  publication  [127]. 

Section  -  3.9  :  Pipelined  Look-Ahead  Implementation  of  a  Class  of  2-D  HR  Filters  :  In  Section-3.7, 
we  have  presented  a  new  scheme  (referred  to  as  distributed  look-ahead )  which  is  a  compromise  between  the  two 
existing  look-ahead  approaches  for  high  speed  implementation  of  1-D  Recursive  Digital  filters.  To  date  neither 
the  Scattered  Look-ahead  nor  the  Distributed  scheme  has  so  far  been  utilized  for  2-D  HR  filter  implementation, 
primarily  because  the  1-D  stability  properties  of  these  LA  schemes  do  not  necessarily  translate  to  general  2-D 
HR  filters.  The  primary  focus  of  this  paper  is  to  demonstrate  that  for  a  special  but  very  important  class  of  2-D 
HR  filters,  namely  for  Denominator  Separable  configurations,  the  benefits  of  these  stable  look-ahead  schemes  can 
indeed  be  taken  advantage  of.  The  results  will  be  submitted  for  review  to  ASILOMAR-97  which  will  be  held  in 
November  at  Naval  Postgraduate  School  [128].  A  detailed  version  is  also  under  preparation  for  a  Journal  paper 

[131]- 


Journal  and  Conference  Articles  -  Published/accepted/under  review 

[11]  A.  K.  Shaw  and  S.  Pokala,  “A  Structured  Matrix  Approach  for  Spatial  Domain  Design  of  2-D  HR  Filters,” 
IEEE  Transactions  on  Circuits  and  Systems ,  accepted  for  publication  Aug.,  1996. 

[12]  A.  K.  Shaw  and  S.  Pokala,  “Spatial  Domain  Design  of  Denominator  Separable  Multidimensional  HR  Filters,” 
Multidimensional  Systems  and  Signal  Processingy  under  2nd  review,  Sep.,  1995. 

[13]  A.  K.  Shaw,  “Optimal  Design  of  Digital  HR  Filters  by  Model-Fitting  Frequency  Response  Data,”  IEEE 


8 


Transactions  on  Circuits  and  Systems  II:  Analog  and  Digital  Signal  Processing ,  vol.  42,  no.  11,  pp.  702-710, 
Nov.,  1995. 

[14]  P.  Misra  and  A.  K.  Shaw,  “Shaping  Time  Response  by  State  Feedback  in  Minimum-Phase  Systems,”  Journal 
of  Control ,  Guidance,  and  Dynamics ,  vol.  18,  no.  4,  pp.  913-916,  Jul.-Aug.,  1995. 

[15]  A.  K.  Shaw,  “Maximum  Likelihood  Estimation  of  Multiple  Frequencies  with  Constraints  to  Guarantee  Unit 
Circle  Roots,”  IEEE  Transactions  on  Signal  Processing,  vol.  43,  no.  3,  pp.  796-799,  Mar.,  1995. 

[16]  A.  K.  Shaw,  “Design  of  Denominator  Separable  2-D  IIR  Filters,”  Signal  Processing,  Switzerland,  vol.  42,  no. 
1,  pp.  191-206,  Feb.,  1995. 

[17]  A.  K.  Shaw,  P.  Misra  and  R.  Kumaresan,  “Multi-Dimensional  System  Identification  From  Impulse  Response 
Data,”  Circuits,  Systems  and  Signal  Processing  -  Special  Issue  on  Multivariable  Systems ,  vol.  13,  no.  6,  pp. 
759-782,  Dec.,  1994. 

[18]  A.  K.  Shaw,  “A  Decoupled  Approach  for  Optimal  Estimation  of  Transfer  Function  Parameters  from  Input- 
Output  Data,”  IEEE  Transactions  on  Signal  Processing ,  vol.  42,  no.  5,  pp.  1275-1278,  May,  1994. 

[19]  A.  K.  Shaw,  “Optimal  Estimation  of  the  Parameters  of  All-Pole  Transfer  Functions,”  IEEE  Transactions  on 
Circuits  and  Systems ,  vol.  41,  no.  2,  pp.  140-150,  Feb.,  1994. 

[110]  A.  K.  Shaw  and  W.  Xia,  “Minimum-Norm  Method  Without  Eigendecomposition,”  IEEE  Signal  Processing 
Letters ,  vol.  1,  no.  1,  pp.  12-14,  Jan.,  1994. 

[111]  A.  K.  Shaw,  “Optimal  Identification  of  Discrete-Time  Systems  from  Impulse  Response  Data,”  IEEE  Trans¬ 
actions  on  Signal  Processing ,  vol.  42,  no.  1,  pp.  113-120,  Jan.,  1994. 

[112]  S.  Pokala,  A.  K.  Shaw  and  M.  Imtiaz,  “Optimal  Least-Squares  Design  of  Pipelined  Recursive  Filters  in 
the  Time-Domain,”  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing ,  Atlanta, 
Georgia,  May,  1996. 

[113]  A.  K.  Shaw  and  M.  Imtiaz  “New  Look-Ahead  Algorithm  for  Pipelined  Implementation  of  Recursive  Digital 
Filters,”  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing ,  Atlanta,  Georgia,  May, 
1996. 

[114]  A.  K.  Shaw  and  M.  Imtiaz  “A  General  Look-Ahead  Algorithm  for  Pipelining  IIR  Filters,”  International 
Symposium  on  Circuits  and  Systems ,  Atlanta,  Georgia,  May,  1996. 

[115]  M.  Imtiaz  and  A.  K.  Shaw  “Tracking  of  Multiple  Targets  using  Pipelined-Adaptive  Algorithm,”  International 
Symposium  on  Circuits  and  Systems ,  Atlanta,  Georgia,  May,  1996. 

[116]  S.  Pokala  and  A.  K.  Shaw,  “Optimal  Spatial  Domain  Design  of  2-D  IIR  Filters,”  IEEE  International  Sym¬ 
posium  on  Circuits  and  Systems ,  Seattle,  Washington,  pp.  1335-1338,  April,  1995. 

[117]  S.  Pokala  and  A.  K.  Shaw,  “Optimal  Frequency  Domain  Design  of  Denominator  Separable  Two-Dimensional 
Digital  IIR  Filters,”  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing ,  Detroit, 
Michigan,  pp.  2133-2136,  May,  1995. 

[118]  A.  K.  Shaw  and  W.  Xia,  “DFT-Based  Preprocessing  for  High-Resolution  Angles-of- Arrival  Estimation  With¬ 
out  Eigendecomposition,”  Twenty- Seventh  ASILOMAR  Conference  on  Signals,  Systems  and  Computers , 
Pacific  Grove,  CA,  pp.  826-830,  Oct.,  1994. 

[119]  A.  K.  Shaw  and  W.  Xia,  “High-Resolution  Angles  of  Arrival  Estimation  using  Minimum-Norm  Method 
Without  Eigendecomposition,”  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing , 
Adelaide,  Australia,  vol.  IV,  pp.  233-236,  April,  1994. 


9 


[120]  A.  K.  Shaw  and  S.  Kundu,  “Improved  ARMA  Modeling  from  Noisy  Observations,”  Twenty-Seventh  ASILO - 
MAR  Conference  on  Signals,  Systems  and  Computers ,  Pacific  Grove,  CA,  Oct.,  1993. 

[121]  A.  K.  Shaw  and  S.  Kundu,  “AR-Spectrum  Estimation  from  Noisy  Observation  Data,”  Twenty-Seventh 
ASILOMAR  Conference  on  Signals ,  Systems  and  Computers ,  Pacific  Grove,  CA,  Oct.,  1993. 

[122]  A.  K.  Shaw,  “Optimal  Design  of  Digital  IIR  Filters  by  Model-Fitting  Frequency  Response  Data,”  IEEE 
International  Symposium-  on  Circuits  and  Systems ,  Chicago,  IL,  pp.  475-478,  May,  1993. 

[123]  A.  K.  Shaw,  “Optimal  Estimation  of  AR-Model  Parameters  from  Impulse  Response  Data,”  31st  IEEE  Con¬ 
ference  on  Decision  and  Control ,  Tucson,  AZ,  pp.  903-908,  Dec.,  1992. 

[124]  A.  K.  Shaw,  “A  New  Algorithm  for  Optimal  Estimation  of  Plant  Parameters  from  Input-Output  Data,”  31st 
IEEE  Conference  on  Decision  and  Control ,  Tucson,  AZ,  pp.  1684-1685,  Dec.,  1992. 

[125]  A.  K.  Shaw  and  W.  Xia,  “Superresolution  without  Eigendecomposition  :  Method  and  Perturbation  Analysis,” 
Under  Preparation  for  IEEE  Transactions  on  Aerospace  and  Electronic  Systems. 

[126]  A.  K.  Shaw  and  M.  Imtiaz,  “Distributed  Look-Ahead  :  A  General  Approach  for  Pipelining  Recursive  Digital 
Filters,”  Under  Preparation  for  IEEE  Transactions  on  Circuits  and  Systems . 

[127]  A.  K.  Shaw  and  M.  Imtiaz,  “Optimal  Least-Squares  Design  of  Pipelined  Recursive  Filters  in  the  Time- 
Domain,”  Under  Preparation  for  IEEE  Transactions  on  Circuits  and  Systems . 

[128]  A.  K.  Shaw  and  M.  Imtiaz,  “Look-Ahead  Pipelined  Implementation  of  a  Class  of  2-D  IIR  Filters,”  Under 
Preparation  for  Thirty-First  ASILOMAR  Conference  on  Signals,  Systems  and  Computers ,  Pacific  Grove, 
CA,  Oct.,  1997. 

[129]  A.  K.  Shaw  and  M.  Imtiaz,  “Pipelined- Adaptive  Tracking  of  Multiple  Sinusoidal  Frequencies,”  Under  Prepa¬ 
ration  for  IEEE  Signal  Processing  Letters. 

[130]  S.  Pokala  and  A.  K.  Shaw,  “Frequency  Domain  Design  of  Two-Dimensional  Digital  IIR  Filters  :  Denominator 
Separable  Case,”  Under  Preparation  for  IEEE  Transactions  on  Circuits  and  Systems. 

[131]  A.  K.  Shaw  and  M.  Imtiaz,  “Pipelined  Look-Ahead  Implementation  of  Denominator-Separable  2-D  IIR 
Filters,”  Under  Preparation  for  IEEE  Transactions  on  Circuits  and  Systems. 


Personnel  Invloved 

Dr.  Arnab  K.  Shaw  (PI) 

Mr.  Wei  Xia  (Graduate  Student) 

Mr.  Srikanth  Pokala  (Graduate  Student) 


10 


CHAPTER  2 


THE  DIGITAL  MICROWAVE  RECEIVER  DESIGN  PROBLEMS 


Introduction 

Digital  processing  of  microwave  signals  in  Electronic  Warfare  (EW)  environment  poses  a  great  challenge  to 
researchers  in  Signal  Processing.  Along  with  the  standard  requirements  of  any  conventional  radar,  EW  receiver 
design  problem  is  complicated  by  the  fact  that  no  knowledge  about  the  input  signal  is  available  to  the  receiver. 
The  nature  of  the  problem  also  requires  that  measurements  and  decisions  be  taken  immediately  or  within  a 
few  seconds  in  an  entirely  passive  mode  of  operation.  All  microwave  receivers  used  in  practice  utilize  analog 
signal  processing  techniques.  The  frequency-band  of  the  EW  signals  are  in  the  GHz  range  and  the  signals  have 
wide  bandwidths  which  necessitate  sampling  and  processing  of  a  massive  amount  of  data  at  or  near  real-time. 
Presently,  no  EW  receiver  processes  microwave  radar  signals  entirely  in  the  digital  domain.  But  it  is  expected 
that  with  the  emergence  of  increasingly  faster  and  inexpensive  digital  computers  and  high-speed  A/D  converters, 
digital  processing  of  microwave  signals  would  most  certainly  be  the  way  of  the  future. 

In  the  past  two  decades,  many  classes  of  radar  and  sonar  receivers  have  been  converted  from  conventional 
analog  technology  to  purely  digital  or  hybrid  systems,  but  EW  receivers  are  yet  to  make  such  a  transition.  The 
primary  technological  factors  that  have  been  holding  back  possible  fabrication  of  any  digital  EW  receiver  are 
probably  twofold.  Firstly,  if  Analog-to-Digital  (A/D)  converters  are  to  be  used  at  the  operating  frequency  range, 
then  the  Nyquist  rate  would  necessitate  sampling  at  the  GHZ  range  and  secondly,  the  digital  hardware  or  firmware 
must  have  the  capacity  to  process  such  high  data  rate  and  produce  effective  results  at  near  real-time. 

Digital  EW  receivers  can  be  expected  to  offer  some  major  advantages  over  their  analog  counterparts.  Foremost 
among  these  is  the  almost  lossless  storage  capability  of  digital  memories  which  can  eliminate  the  dependence  on 
lossy  analog  delay  lines.  Digital  processors  and  memory  chips  are  relatively  inexpensive,  compact  in  size  and  have 
low  weight  and  the  trends  are  towards  even  further  reductions.  Digital  signal  processing  algorithms  and  digital 
computing  technology  have  matured  tremendously  and  offer  a  wide  range  of  capabilities.  Parallel  processing, 
pipelining,  RISC,  VLSI  design,  systolic  architecture,  vectorization  and  array  processing,  fault  tolerant  computing 
and  etc.,  are  only  some  of  the  well-known  aspects  of  digital  computing  that  the  last  few  decades  of  research  have 
produced.  As  our  research  progresses,  we  intend  to  study  if  some  of  these  ideas  can  be  incorporated  in  the  digital 
receiver  in  order  to  improve  the  efficiency  and  accuracy  of  its  performance. 

The  primary  task  of  a  microwave  receiver  is  to  gather  data  for  sorting  of  signals  and  for  identifying  the 
radar-type.  Based  on  these  information,  jamming,  weapon  delivery  or  other  options  are  considered.  In  order  to 
perform  these  tasks,  the  receiver  must  analyze  the  received  radar  pulses  and  measure  or  estimate  the  following 
six  parameters  :  Angle-of- Arrival  (AOA),  Radio  Frequency  (RF),  Time  of  Arrival  (TOA),  Pulse  Amplitude  (PA), 
Pulse  Width  (PW)  and  Polarization  (P). 

A  critical  requirement  of  an  EW  receiver  is  the  AOA  measurement  which  is  known  to  be  a  rather  difficult 
multidimensional  nonlinear  optimization  problem,  especially  when  multiple  closely-spaced  threats  are  to  be  re¬ 
solved.  It  is  also  desirable  to  have  high  sensitivity  and  large  dynamic  range  such  that  a  broad  range  of  signals, 
including  weak  ones,  can  be  detected. 

As  part  of  this  project  several  AOA/frequency  estimation  algorithms  has  been  developed  and  studied.  Most 
existing  high-resolution  frequency-estimation  algorithms  rely  on  special-purpose  hardware  or  software,  such  as, 
Eigendecomposition  or  SVD.  In  Section  2.1,  a  DFT-based  Minimum-Norm  method  is  proposed  which  does  not 
require  any  eigendecomposition  but  produces  high-resolution  frequency  estimates.  This  new  algorithm  needs 
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only  to  compute  the  DFT  of  the  Autocorrelation  matrix  to  separate  the  signal  and  noise  subspaces.  Hence  the 
computational  burden  is  much  lower  than  existing  high-resolution  methods.  Therefore,  this  algorithm  appears 
to  be  very  well-suited  for  EW  applications.  The  Statistical  Performance  Analysis  of  this  new  algorithm  has  also 
been  performed  and  the  results  are  included  in  Section  2.1  also. 

Another  new  class  of  algorithms,  referred  to  as  KiSS/IQML,  have  been  developed  recently  for  obtaining 
the  Maximum  Likelihood  Estimates  (MLE)  of  frequencies  or  AOAs  from  the  roots  of  z— polynomials.  But  the 
estimated  roots  are  not  guaranteed  to  fall  on  the  unit  circle,  as  desired.  Based  on  the  theory  on  zeros  of 
polynomials,  a  new  scheme  is  proposed  in  Section  2.2  here  that  will  ensure  unit  circle  roots.  Many  frequency 
estimation  methods  make  use  of  the  property  that  a  sinusoidal  process  can  be  modeled  as  a  limiting  case  of  a 
narrow-band  auto-regressive  (AR)  process.  But  the  performance  of  all  existing  AR  parameter  estimation  methods 
degrade  significantly  when  the  observation  data  is  corrupted  with  noise.  A  pre-filtering  approach  is  presented 
in  Section  2.3  that  can  improve  AR-parameter  estimates  from  noisy  observation  data.  Another  data-adaptive 
approach  for  improved  modeling  of  ARM  A  processes  from  noisy  observations  is  given  in  Section  2.4. 

Parameter  estimation  schemes  either  follow  or  work  in  parallel  with  a  detection  scheme  ensuring  the  presence 
of  any  threat.  A  combined  detection-estimation  scheme  has  the  potential  to  cut-down  computational  burden  on 
the  signal  processor.  As  a  part  of  this  project,  statistical  theory  on  hypothesis  testing  has  been  utilized  for 
detecting  whether  a  threat  is  present  or  not.  In  Section  2.5,  the  time-domain  detection  problem  has  been 
presented  for  single  and  multiple  samples.  Specifically,  the  detection  thresholds  and  Probability  of  Detection 
based  on  Neyman-Pearson  Criterion  have  been  derived. 

The  AOA  estimation  algorithms  presented  in  Sections  2. 1-2.2  work  on  batch  mode  where  the  targets  are 
assumed  to  be  "locally  stationary”.  However,  in  many  practical  situations  the  targets  may  be  non-stationary  an 
it  is  desirable  to  track  its  movements  adaptively.  In  these  regards,  Pipelined-Adaptive  algorithms  are  proposed 
in  Section  2.6  for  tracking  multiple  Frequencies  or  Angles-of- Arrival  (AOA)  of  moving  targets.  Pipelining  of 
adaptive  filters  pose  a  critical  challenge  because  of  the  timing  mismatch  arising  from  the  feedback  signals.  In  this 
work,  some  relaxation  techniques  have  been  utilized  to  pipeline  adaptive  algorithms  for  high-speed  tracking  of 
frequency /AO  As. 
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Section  -  2.1  :  SUPERRESOLUTION  WITHOUT  ElGENDECOMPOSITION  :  METHOD  AND 

Perturbation  Analysis 


Summary 

Many  existing  methods  for  estimating  closely  spaced  sinusoidal  frequencies  utilize  the  eigenvectors  of  Auto¬ 
correlation  (AC)  matrices  [1,  10,  15,  26,  30].  Instead,  this  work  considers  the  use  of  the  DFT  of  AC  matrices 
(DFT-of- AC)  for  extracting  the  signal  and  noise  subspaces.  When  the  signal-subspace  part  among  the  DFT-of-AC 
vectors  are  used  in  the  Minimum-Norm  method  (MNM)  framework,  almost  identical  high-resolution  frequency 
estimates  are  produced.  Theoretical  Perturbation  Analysis  of  the  proposed  DFT-based  MNM  (D-MNM)  has  also 
been  carried  out.  The  analysis  confirms  that  the  estimates  are  theoretically  unbiased  and  have  lower  theoretical 
Mean-Squared  Error  indicating  improved  high-resolution  performance,  especially  at  low  SNR.  The  primary  advan¬ 
tages  of  extracting  signal/noise  subspace  information  from  DFT-of-AC  are  reduced  computational  and  hardware 
complexity  than  existing  methods  that  need  to  perform  the  Eigendecomposition  iteratively. 

I :  Introduction 

In  many  important  practical  applications,  such  as  radar,  sonar  and  astronomy  etc.,  the  resolution  capability 
of  FFT  is  inadequate.  Overcoming  the  resolution  limitation  of  DFT  has  been  a  vigorously  researched  topic 
in  Signal  Processing  in  the  past  three  decades.  The  modern  methods  attain  the  desired  ‘High-Resolution’  or 
‘Superresolution’  at  the  cost  of  considerable  computational  burden.  The  existing  well-known  methods  often  utilize 
Eigen-Decomposition  (ED),  Singular  Value  Decomposition  (SVD)  or  Maximum  Likelihood  (ML)  method  which 
is  based  on  nonlinear  optimization  [1-5,  8-10,  14-20,  22-24,  26,  29,  30,  32-37,  42-45,  48,  49].  These  algorithms, 
though  highly  effective,  can  only  be  implemented  iteratively  because  of  their  inherent  nonlinearity,  which  limits 
their  real-time  capabilities. 

The  primary  objective  of  this  paper  is  to  effectively  combine  the  computational  simplicity  of  DFT  with  the 
underlying  mathematical  philosophy  of  certain  high-resolution  methods.  The  desired  goal  is  to  achieve  high- 
resolution  without  any  iterative  optimization.  Specifically,  many  existing  high-resolution  techniques,  such  as  the 
Minimum-Norm  method  (MNM),  extract  the  signal  and  noise  subspace  information  from  the  eigenvectors  of  the 
Autocorrelation  (AC)  matrices  [15,  26].  It  is  shown  in  this  paper  that  the  DFT  of  the  AC-matrix  (DFT-of-AC) 
essentially  performs  an  equivalent  task  of  extracting  and  decoupling  the  signal  and  noise  subspace  information. 
Hence,  it  is  proposed  that  the  signal  eigenvectors  be  replaced  by  the  largest-norm  DFT-of-AC  vectors.  It  is 
demonstrated  that  when  the  DFT-of-AC  vectors  with  larger  norms  are  used  in  the  MNM  framework,  mostly 
better  or  almost  equivalent  high-resolution  DOA  estimates  are  produced.  The  bias,  mean-squared  error  and 
the  root  locations  of  the  proposed  DFT-based-MNM  (D-MNM)  also  compare  well  with  the  Eigendecomposition- 
based  MNM  (E-MNM).  The  simulations  further  show  that  the  high-resolution  performance  of  the  D-MNM  is 
more  robust  at  low  SNR. 

In  order  to  establish  theoretical  justification  of  the  performance  of  the  D-MNM  algorithm,  we  have  also 
conducted  theoretical  Perturbation  Analysis  of  the  estimates  produced  by  the  algorithm.  The  theoretical  study 
corroborates  closely  with  the  superior  performance  already  observed  in  simulations.  The  details  of  the  following 
are  included  after  introducing  the  method.  Firstly,  the  Bias  and  the  Mean-Squared-Error  (MSE)  in  the  estimates 
of  the  AOAs  are  shown  to  be  linearly  related  to  the  Bias  and  MSE  in  the  roots  of  the  D-MNM  polynomial 
which,  in  turn,  are  shown  to  depend  on  the  Bias  and  MSE  for  the  D-MNM  coefficient  vector.  Then  the  statistics 
of  the  coefficient  error  are  related  to  those  of  the  observed  data  and  the  AC  matrix  estimate.  Finally,  all 
the  intermediate  results  are  combined  and  utilized  to  find  the  direct  statistical  relationship  between  the  AOA 
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errors  and  the  observations.  The  theoretical  results  indicate  that  the  high-resolution  performance  of  D-MNM  is 
uniformly  superior  than  its  eigen-based  counterpart,  especially  at  low  SNR.  The  performance  is  also  superior  than 
the  eigen-based  root-MUSIC  method  at  low  SNR.  Furthermore,  D-MNM  appears  to  provide  better  success  rate 
among  all  methods  at  low  SNR.  The  theoretical  analysis  closely  follows  the  performance  with  simulated  data, 
which  verifies  the  validity  of  the  formulae  derived  here  theoretically. 

The  major  significance  of  the  proposed  algorithm  is  that,  no  complicated  iterative  optimization  is  needed  and 
the  signal-subspace  information  is  extracted  only  by  a  single  matrix  multiplication.  Hence,  hardware  implementa¬ 
tion  of  D-MNM  for  real-time  high-resolution  AOA/Frequency  estimation  may  be  feasible  with  currently  available 
technology.  It  may  be  noted  here  that  results  on  some  preliminary  simulations  on  the  DFT-based  method  was 
presented  in  [38,  39],  though  no  performance  analysis  was  available  at  the  time. 

The  paper  is  organized  as  follows.  In  Section  II,  the  AOA  estimation  problem  is  defined  and  in  Section  III 
some  existing  approaches  are  discussed.  Some  useful  properties  of  the  AC  matrix  are  given  in  Section  IV.  Then  in 
Section  V,  the  proposed  D-MNM  algorithm  is  described.  In  Section  VI,  the  details  of  the  Perturbation  analysis  of 
D-MNM  are  presented.  Some  simulation  results  are  given  in  Section  VII  and,  finally  the  paper  is  wrapped  with 
some  concluding  remarks  in  Section  VIII. 

II :  Problem  Definition 

This  paper  addresses  the  problem  of  estimating  of  the  Directions  of  Arrival  (DOA)  of  densely  spaced  narrow- 
band  targets.  Suppose  that  p  plane  waves  originating  from  far-field  point  sources  at  distinct  directions  impinge 
on  a  linear  array  of  N  equally  spaced  sensors.  The  signal  sampled  simultaneously  at  mth  instant  of  time  at  N 
equally  spaced  sensors  form  a  ‘snapshot’  vector  defined  as, 

xm  A  [xm(Q)  xm(l)  ...  xm(N-  1)]*.  (71.1) 

In  the  presence  of  noise,  the  observation  samples  can  be  written  as, 

xm(n)  =  xm(n)  +  zm(n)  (77.2) 

where,  zm(n)  represents  the  additive  observation  noise  and/or  the  modeling  error  and  xm(n )  denotes  the  signal 
part  of  the  observation,  which  is  given  by 

p 

xm(n)  =  J2Am(i)ejZfL(n  “  ^)sin<?’  +  n  =  0, 1, ....  AT  —  1  (//.3) 

t  =  l 

Number  of  narrowband  sources  present 
Spacing  between  sensor  elements 
Wavelength  of  radiation  of  the  received  signals 
Direction-of- Arrival  (DOA)  of  the  iih  source 
Amplitude  of  the  ith  source  at  the  mth  snapshot 
Phase  angle  of  the  ith  source  at  the  mih  snapshot, 

Uniformly  distributed  between  —  7r  and  7r. 

The  noise  zm{n)  is  assumed  to  be  zero-mean  and  uncorrelated  with  the  source  signals  and  it  has  a  variance  of  a2. 
The  signal  model  can  be  written  in  a  more  succinct  form  as, 

xm(n)  =  (//.4) 

8  =  1 


where, 

P 

d 

A 

0i 

A, „ 


.(0 

(i) 
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where,  and  Aim  are  defined  as 


a  27rd  . 

coi  A  — —  smdi  and 


Aim  A  Am(i)e~j2^i(!^±^sinei  + 


(II- 5) 


Further  details  about  the  above  model  may  be  found  in  [6].  With  the  above  formulation  the  model  for  the 
observation  matrix  can  be  written  as, 

X  A  TA 


where, 


T  A 


pj^i 


1 

e^2 


e;u;i(JV-l)  ejw2(N-l) 

A  [ti  t2  ...  tp], 

A  A  [ ai  a2  ...  a m  ]  and 


1 

jwp(N-l) 


a  m  A 


r  Aim 

A2m 


la 


■pm 


for  m  —  1, 2, . . . ,  M. 


(II- 7) 

(II- 8) 

(J/.9) 

(77.10) 

(77.11) 


For  half  wavelength  spacing  between  two  successive  sensors  of  the  line  array,  w*  =  7r  sin^j.  With  M  snapshot 
vectors  defined  in  (//.2),  the  N  x  M  observation  matrix  X  is  formed  as, 


X  A  [xi  x2  ...  xM]- 

Using  the  observation  matrix,  the  spatial  covariance  matrix  can  be  estimated  as, 

C  A  ^(XX») 


t  7f  E  x-x-- 


(77.12) 

(77.13a) 

(77.136) 


m=l 


The  description  of  the  observation  and  the  model  is  now  complete.  Given  the  noisy  observation  matrix  X,  the 
problem  under  consideration  in  this  paper  is  to  estimate  the  u>t-’s  and  Aim  s.  Note  that  the  complex  amplitudes 
can  be  estimated  linearly  once  the  u>As  are  known  but  the  estimation  of  u^-s  poses  the  greatest  difficulty  because 
it  is  a  highly  nonlinear  optimization  problem. 


Ill  :  Existing  Methods 

It  is  apparent  from  the  problem  statement  that  the  DOA  ( 0t- )  estimation  problem  is  mathematically  equivalent  to 
the  Frequency  Estimation  (u>2-)  problem  which  has  been  a  major  research  topic  in  many  areas  of  science.  Indeed, 
in  the  last  couple  of  hundred  years,  the  search  for  diidden  periodicities’  from  observed  data  has  appeared  in  varied 
forms  in  several  seemingly  differing  disciplines  of  science. 

III. a  :  The  Periodogram  and  its  Resolution  Limitation 
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Ever  since  its  discovery  in  1965  [6],  the  FFT  has  been  the  primary  tool  for  estimating  Directions  of  Arrival 
(DO A)  or  frequencies  of  far-field  sources  from  noisy  observation  data.  The  software  or  hardware  implementation 
of  FFT  is  remarkably  straight-forward.  To  date,  the  periodogram  continues  to  be  the  most  frequently  used 
method  for  frequency/DOA  estimation  [21,  27].  In  fact,  it  is  well  known  that  for  localizing  a  single  target,  if 
the  noise  in  the  observed  data  is  Gaussianly  distributed,  the  periodogram  [27]  produces  the  maximum  likelihood 
estimate.  But  in  case  of  multiple  targets,  the  periodogram  cannot  resolve  two  frequencies  which  are  separated 
by  less  than  the  bin-width  of  the  FFT.  In  fact,  when  the  sources  are  spaced  at  less  than  the  DFT  bin-width, 
the  periodogram  fails  to  distinguish  two  closely  spaced  frequencies  and  only  provides  a  single  frequency  estimate 
instead  of  two.  The  last  statement  truly  portrays  the  problem  one  faces  while  resolving  two  closely  spaced  sinusoids 
when  a  relatively  short  data  record  is  available.  Clearly,  if  any  amount  of  data  is  available  for  processing,  the 
periodogram  of  sufficiently  Zero-padded  (and  possibly  Windowed)  data  will  provide  reasonably  good  estimates. 
But  in  many  problems  of  practical  interest  only  short  data  record  is  available  and  one  has  to  overcome  the 
periodogram’s  resolution  limitation  by  resorting  to  what  are  commonly  known  in  the  signal  processing  literature 
as  'High- Resolution’  or  'Superresolution ’  techniques.  The  major  contributions  in  the  higher  resolution  approaches 
are  highlighted  next. 

Ill.b  :  High-Resolution  Methods 

A  multitude  of  DOA/Frequency  Estimation  algorithms,  their  variations  and  analysis  are  available  in  the 
literature  [1-5,  7-20,  22-26,  28-30,  32-49].  In  the  following  paragraphs  only  some  of  the  major  developments  are 
briefly  discussed. 

Minimum  Variance  Method  :  This  was  perhaps  the  earliest  high  resolution  methods  which  was  specifically  devel¬ 
oped  for  frequency- wavenumber  estimation.  In  order  to  improve  upon  Periodogram’s  resolution  limit,  Capon  had 
proposed  a  Minimum  Variance  method  which  is  a  linear  estimator  that  minimizes  the  interference  at  frequencies 
outside  the  band  of  interest  [4].  Its  performance  has  been  shown  to  be  better  than  the  periodogram  estimator 
but  worse  than  the  modeling  based  estimators  [20]. 

Model-Based  Methods  :  A  major  motivation  for  many  modern  high-resolution  frequency  estimation  methods  has 
come  from  the  desire  to  achieve  more  exact  models  for  the  sinusoids-in-noise  data.  In  the  Parameter  Estimation 
area  of  statistical  time-series  analysis,  it  had  been  well  established  that  Auto- Regressive  (AR)  modeling  is  very 
appropriate  for  modeling  data  with  peaky  spectra.  But  in  the  frequency  estimation  field  also,  it  had  been  a 
common  knowledge  that  data  composed  of  sinusoids  in  noise  tend  to  have  peaky  spectra.  Consequently,  frequency 
estimation  based  on  AR-modeling  has  received  considerable  attention  [3,  9,  24,  29]. 

Depending  on  how  the  autocorrelation  values  are  estimated,  there  are  three  types  of  AR  parameter  estimation 
methods,  namely,  Autocorrelation  method,  Covariance  method  and  Modified  Covariance  method  (also  known  as 
the  Forward-Backward  method).  The  later  two  cases  are  more  appropriate  for  sinusoidal  processes  because  of 
their  implicit  relationship  with  Prony’s  method  which  provides  perfect  frequency  estimates  when  no  noise  is 
present.  Incidentally,  the  Maximum  Entropy  method  proposed  by  Burg  [7]  and  the  Linear  Prediction  based 
spectral  estimator  [9]  both  produce  essentially  identical  frequency  estimates  as  the  Covariance  method. 

When  p  sinusoids  are  present  and  a  pth  order  AR  model  is  used,  the  frequency  estimates  are  found  to  be 
poor  at  low  SNR  (<  30 dB).  To  circumvent  this  hurdle,  larger  order  ( L  >  p)  AR  model  has  been  proposed  [25, 
60].  The  larger  model  order  tends  to  accommodate  a  major  part  of  the  interfering  noise  and  thereby  reduces  the 
effect  of  noise  in  the  estimates.  The  larger-order  approach  performs  poorly  below  20dB  SNR. 
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Eigen- Analysis  of  the  Auto- Correlation  Matrix  of  Sinusoid-in-N  oise  Data  :  AR  modeling  based  approaches  offer 
better  resolution  performance  than  their  predecessors  but  these  as  well  as  the  earlier  methods  are  basically 
general  spectrum  estimation  methodologies  applied  to  this  specific  narrow-band  problem.  Since  the  mid-to-late 
seventies,  a  whole  new  class  of  algorithms  are  being  developed  by  effective  exploitation  of  the  special  properties 
of  the  autocorrelation  matrix  of  the  sinusoids-in-noise  data.  For  N  —  pA  1,  the  eigendecomposition  of  C  was  first 
utilized  by  Pisarenko  [23]  who  showed  that  the  ^-polynomial  formed  with  elements  of  the  eigenvector  corresponding 
to  the  smallest  eigenvalue  has  roots  at  the  signal  frequencies.  Though  the  idea  is  elegant,  Pisarenko’s  method 
performs  quite  poorly  for  noisy  signals.  Pisarenko’s  approach  was  later  improved  upon  by  Kumaresan  [15]  where, 
for  N  >  p  cases,  all  the  noise  eigenvectors  had  been  utilized.  As  an  alternate  approach,  it  was  shown  in  [15]  that 
the  signal  subspace  eigenvectors  can  also  be  utilized  to  form  a  noise  subspace  vector  which  should  have  zeros  at 
the  signal  frequency  locations.  This  was  achieved  in  [15,  26]  by  formulating  a  Minimum-Norm  criterion  which  is 
the  framework  that  will  be  used  in  the  proposed  work. 

Another  major  improvement  on  Pisarenko’s  approach  was  presented  by  Schmidt  [30]  and  Bien venue  and 
Kopp  [1].  They  proposed  to  combine  the  eigenvectors  corresponding  to  the  ( L  —  p)  smaller  eigenvalues  of  C  and 
used  an  orthogonality  criterion  to  obtain  the  frequency  estimates.  In  the  literature,  this  approach  is  known  as 
the  ‘MUSIC’  method. 

It  may  be  pertinent  to  emphasize  here  that  the  approach  proposed  in  this  work  for  extracting  signal  or  noise 
subspace  ‘without  eigendecomposition’  may  be  combined  with  either  the  MNM  or  the  MUSIC  framework.  The 
MNM  framework  has  been  preferred  in  the  development  in  Section  V  because  in  case  of  the  Minimum-Norm 
method,  the  frequencies  are  found  directly  from  the  polynomial  roots.  On  the  other  hand,  a  search  procedure 
is  necessary  in  case  of  MUSIC  for  estimating  the  frequencies.  The  polynomial  version  of  MUSIC,  known  as 
‘root-MUSIC’,  could  also  be  used  but  in  that  case  the  order  of  the  ^-polynomial  would  be  twice  that  of  MNM. 

Maximum-Likelihood  Method  :  This  class  of  algorithms  maximize  the  likelihood  function  for  the  observed  data, 
leading  to  optimization  of  a  non-linear  criterion  which  can  only  be  performed  iteratively.  Several  different  ap¬ 
proaches  are  available  in  the  literature  [16-18,  27-29,  33-35,  37,  48]  among  which  the  recently  proposed  Constrained 
MLE  approach  developed  [37]  by  the  first  author  appears  to  offer  the  most  accurate  results. 

Other  Methods  and  the  Motivation  for  the  Proposed  Method  :  As  listed  in  the  references,  there  are  a  large  number 
other  methods  that  address  the  high-resolution  Frequency/DOA  estimation  problems.  In  order  to  achieve  the 
desired  high-resolution  capability,  all  these  algorithms  utilize  some  form  of  eigen- analysis  or  non-linear  optimiza¬ 
tion,  both  of  which  are  computationally  intensive  for  real-time  applications.  The  primary  objective  of  this  paper 
is  to  study  whether  the  computational  simplicity  of  DFT  can  be  effectively  combined  with  the  underlying  math¬ 
ematical  framework  of  some  of  the  existing  high-resolution  methods.  The  final  goal  is  to  achieve  high-resolution 
without  any  iterative  optimization  such  that  real-time  implementation  may  be  feasible  with  existing  hardware. 
The  proposed  method  makes  use  of  the  special  properties  of  correlation  matrices  which  are  outlined  next. 

IV  :  Some  Properties  of  the  Autocorrelation  Matrix 

Since  the  data  described  by  (J/.3)  is  uncorrelated,  zero  mean  WSS  process,  the  N  x  N  (N  >  p)  covariance 
matrix  C  will  have  the  following  matrix  decomposition  when  there  is  no  observation  noise, 

c  =  T£Tff  (IV.  1) 

where,  E  A  diag  (af  a\  ...  <7p)  and  denotes  the  power  of  the  i-th  signal.  Note  that  this  ideal  C  has  rank 
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p.  In  this  case,  the  eigen-decomposition  of  C  can  be  written  as, 
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For  observations  with  noise  as  defined  in  (17.3) , 

C  =  TET*  +  ah. 


(IV.  3) 


Note  that  this  theoretical  C  has  rank  N  though  the  signal  part,  TET^  has  rank  p.  In  this  case,  the  eigen- 
decomposition  of  C  can  be  written  as, 


CV  =  [(Ai  +  <j2)vi  •  •  •  (Ap  +  a2)vp  a2vp+1  •  •  •  a2\N  ]  (IV A) 

where,  the  A,-’s  and  a2  represent  the  signal  and  noise  eigenvalues.  But  in  practice,  the  eigendecomposition  has  to 

be  performed  on  the  sample  covariance  matrix  C  as  defined  in  (71.13)  and  then  the  noise  eigenvalues  will  not  be 
equal  but  will  be  absorbed  with  the  signal  eigenvalues  also.  In  that  case, 

CV  =  [Aivi  •••  ApVp  Ap+iVp+i  •••  A  nvn]  (IV.  5) 

where,  the  estimated  eigenvalues  are  ordered  as,  \\  >  A2  >  •••  \n.  The  eigenvectors  corresponding  to  the 
p  largest  eigenvalues  are  called  the  ‘signal  eigenvectors’  which  constitute  the  ‘signal-subspace’.  All  the  other 

(N  —  p )  eigenvectors  are  known  as  the  ‘noise  eigenvectors’.  Note  also  that  the  p  ‘signal  eigenvectors’  of  C  span 

the  subspace  defined  by  the  columns  of  T  and  that  they  are  orthogonal  to  the  ‘noise  subspace’  eigenvectors. 


V  :  The  Proposed  DFT-Based  Minimum-Norm  Method  (D-MNM) 


As  a  significant  departure  from  the  eigen-based  approaches  discussed  in  the  previous  section,  this  work  advocates 
that  the  signal-subspace  information  be  extracted  from  the  DFT-of-AC  matrix  which  can  be  accomplished  with 
a  single  matrix  multiplication.  This  will  eliminate  the  need  for  iterative  calculation  of  eigenvectors  which  is 
computationally  intensive.  The  central  idea  behind  the  DFT-of-AC  matrix  is  analyzed  first. 

V.a  :  Signal  and  Noise  Subspace  Extraction  from  the  DFT-of-AC  Matrix 

Let  the  DFT  matrix  be  denoted  as, 


D  A  [ei  e2  •••  ejv],  (V.l) 

where,  the  elements  of  the  fc-th  DFT-vector  et  is  defined  as,  ej t(/)  =  e^kl,  for  k,l  =  0,1,  2,  . . .,  IV  —  1.  If  the 
frequencies  wts  are  all  on  the  DFT  bins  and  if  there  is  no  observation  noise,  then  in  general, 

fi  A  Ce*;  (V.2a) 
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^  j*Z(xme*)xm,  using  (1. 136) 

m=l 

1  M 

using  (1.8) 


m  =  1 
M 

m=l 


i/ 


tfe*1 


L*^e* 


If  the  fc-th  DFT  vector  ej corresponds  to  one  of  the  u>*  frequencies, 

1  m  1  Af 

f*  =  77  E  ^mTa™  =  Tm  £ 


m=l 


m  =  l 


-  1  A*  A 

'dfci" 

=  T 

bTZLiM 

=  T 

H 

-  Jf  Ern=  1  A-lmAp-m  - 

-dj bp. 

(V.3) 


where,  <7fc/s  denote  the  covariance  of  the  complex  amplitudes.  Assuming  the  number  of  samples  M  to  be  large 
and  since  A*ms  are  independent  random  variables,  ^Akrtl,Aim  A  &ki  =  Hence, 


(V.2  b) 
(V.2c) 

(V.2  d) 


dfctfc  —  djjiefc. 


(V.4) 


Note  that  the  norm  of  f*  is  directly  proportional  to  the  signal  power,  <r|,  z.e.,  this  norm  will  be  large  if  the  signal 
power  is  significant.  On  the  other  hand,  if  a  DFT-vector  e*  does  not  correspond  to  any  of  the  iOi  frequencies  then 
due  to  orthogonality,  =  0,  Vi.  For  such  cases, 


h  =  0.  (V5) 

For  this  ideal  case  then,  the  DFT-of-AC  has  the  following  decomposition, 

F  A  CD  (V.6a) 

A  [fi  f2  ...  fN]  (VM) 

— *  [Aiui  ApUp  0  •••  0]  (V.&c) 


where,  the  AjS  and  uts  are  the  lengths  and  unit  vectors  of  each  fz,  respectively.  Note  that  the  unit  vectors  in  the 
matrix  in  (K6c)  have  been  rearranged  so  that  the  zero/nonzero  components  are  clustered  together.  Interestingly, 
this  decomposition  appears  to  be  very  similar  to  the  usual  Eigendecomposition  of  noiseless  and  ideal  C,  as  given  by 
{V.2).  For  this  ideal  signal  scenario  again,  if  the  DFT-of-AC  is  formed  using  the  theoretical  and  noisy  Covariance 
matrix  of  (V.3),  then  the  decomposition  has  the  form, 

F  =  CD 

=  TST^D  +  D 

-►  [(Ai+<rf)ui  •••  (Ap+<rf)up  <r*up+i  ••• 
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(V.7a) 

(V.76) 

(V.7c) 


where  the  u;’s  have  been  arranged  in  decreasing  order  of  lengths.  Note  again  that  this  decomposition  is  analogous 
to  the  one  in  (V.4).  In  this  case  also,  the  p  largest-norm  vectors  of  the  DFT-of-AC  matrix  contain  the  signal 
subspace  information. 

In  practice,  the  will  not  be  on  the  DFT  bins  and  the  observations  may  also  be  noisy  and  hence,  the 
decomposition  in  (V.6)  or  (V.7)  will  not  hold.  But  the  DFT-components  (f^s)  closer  to  the  signal  frequencies  will 
tend  to  have  larger  norms.  Hence,  for  the  general  scenario,  when  the  observation  data  is  noisy  and  the  angular 
frequencies  are  arbitrarily  spaced,  the  signal/noise  subspace  decomposition  can  be  formed  as  : 


-*•  [Aiui 

A  A[U5 

A  [F 5  | 

A  [fi  f2  . 


Ap+iUp+i 


Ajvuat] 


ip  •  ip+i 


(V.8o) 
(V.8  b) 
(V.8c) 

(V.8  d) 


where,  the  matrix  Fs  is  formed  with  p  number  of  f,  vectors  having  larger  norms,  A,  >  A2  >  •  •  •  >  Ajv  are  the 
norms  of  the  corresponding  f;  vectors  and  the  matrices  A,  Us  and  U n  are  formed  as, 

Al  a,  1  I  -  ii  r  i  -  1 1 

A  A  .  ,  U5  A  ui  u2  . . .  up  and,  UN  A  up+i  ...  .  (V8e) 


It  may  be  observed  again  that  the  decomposition  in  (V.8)  is  analogous  to  the  eigen-based  counterpart  in  (7V.5). 
It  may  also  be  noted  here  that  in  case  of  the  ideal  signal  cases  of  (V.6)  and  (V.7),  an  unit  vector  u*  corresponds  to 
one  of  the  DFT-vector  e*>,  but  in  the  general  case  of  (V.8),  they  are  linear  combinations  of  the  DFT-components 
close  to  the  signal  frequencies. 

V.b  :  Incorporation  of  DFT- Based  Signal  Subspace  in  the  Minimum-Norm  Framework 

The  principal  idea  behind  the  Minimum-Norm  method  is  to  form  an  appropriate  ‘noise-subspace’  vector  d 
which  is  orthogonal  to  the  ‘signal-subspace’  defined  by  F 5.  Let, 


D{z)  A  dkz 


be  an  ( N  —  l)-th  order  ^-polynomial  with  p  zeros  at,  z,-  =  for,  i  =  1,  . . .  ,p,  corresponding  to  the  DOAs. 
The  coefficient  vector  is  denoted  as, 


d  A  [do 


<1n- i]T,  A 


O'- io) 


where,  d0  =  1  and  d'  contains  the  unknown  coefficients.  According  to  the  MNM  philosophy  [15],  if  Fs  does 
constitute  of  the  signal-subspace,  then  d  must  be  orthogonal  to  Fs,  he., 


Ffd  =  0. 


(V.11) 


d  needs  to  be  found  by  solving  this  underdetermined  set  of  equations  which  has  infinite  number  of  solutions. 
According  to  [15,  26],  the  solution  that  also  minimizes  the  norm  ||d||2,  possesses  the  desirable  property  that  all 
its  roots  fall  inside  the  unit  circle.  This  ‘minimum-norm’  solution  of  d  for  solving  (V.ll)  can  be  expressed  as  : 


(V.12a) 


-  G^GG^-'g 
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where,  is  partitioned  as, 


Ff  A  [g  |  G],  (V.l2b) 

Once  d  is  estimated,  the  p  roots  of  D(z)  closest  to  the  unit  circle  are  used  to  find  the  DOAs.  It  may  be  recalled 
that  in  E-MNM  the  signal-subspace  eigenvectors  vj,  V2,  . . vp,  as  defined  in  (V.5)  are  used  to  form  F$  [15, 
26].  But  in  case  of  the  proposed  approach,  no  eigendecomposition  is  necessary.  Post- multiplication  of  C  by  the 
DFT-matrix  D  is  all  that  is  required  to  extract  the  signal  subspace  in  (V.8). 

V.c  :  Summary  of  the  Proposed  D-MNM  Algorithm 


The  key  steps  and  some  alternative  possibilities  are  summarized  in  this  Section. 


V.c.l  :  Algorithm  Steps 


1.  Form  the  Covariance  Matrix  estimate  using  forward-backward  method  [15]  : 

M 

2  M 


1  M 

c  A  —  xmx£  +  X^x^". 


m=l 


(V.13) 


The  backward’  vector  is  defined  as  x^  A  JxJ^,  where,  J  denotes  the  permutation  matrix  with  l’s  at  the 
cross-diagonal  entries  and  *  denotes  the  complex- conjugate  operation. 


2.  Post-multiply  C  by  the  DFT  matrix  D  to  form  the  DFT-OF-AC  matrix,  F  A  CD. 

3.  Form  F 5  as  in  (K8c)  using  the  p  unit  vectors  corresponding  to  the  largest  norms.  Partition  F s  as  in  (V.12 b). 

4.  Estimate  the  d  vector  using  (V.12 a)  and  form  the  D(z)  polynomial  using  the  elements  of  d. 

5.  Find  the  roots  of  D(z).  Pick  the  p  roots  closest  to  the  unit  circle  to  find  the  desired  frequencies/DOAs. 


V.c.2  :  Alternate  Possibilities 


Steps  2  and  3  :  Post-multiplication  of  the  AC-matrix  by  a  DFT-matrix  has  been  used  here  because  the  decom¬ 
positions  as  described  in  Section  V  appear  analogous  to  eigendecomposition.  But  it  is  easy  show  that  identical 
results  can  be  obtained  if  the  AC-matrix  is  pre-multiplied  by  a  DFT  matrix,  i.e.,  the  DFT-of-AC  matrix  can  also 
be  formed  alternately  as,  Fi  A  DC.  In  that  case,  the  largest  norm  row  vectors  of  the  DFT-of-AC  matrix  Fi 
must  be  used  to  form  Fjf  defined  in  (K13). 


Step  4  •  This  step  requires  inversion  of  a  matrix  of  dimension  (N  —  1)  x  (N  —  1).  This  can  be  avoided  by  orthog- 
onalizing  the  p  largest  norm  vectors  in  F 5.  Let,  F£  be  the  new  ‘signal-subspace’  matrix  with  the  orthonormal 
set  of  vectors  which  can  be  written  in  partitioned  form  as, 


FsW  A  [g0  |  G0]. 


With  these  partitioned  matrices,  d  can  again  be  found  in  Step-4  as  [15], 


d  = 


1 

-  Gfg0/(1  -  g?g0) 


(V.14) 


(F.15) 


It  may  be  mentioned  here  that  in  [15],  p  orthonormal  signal  eigenvectors  were  used  to  form  F s,  whereas  here  F£ 
is  formed  by  orthogonalizing  the  p  largest  norm  vectors  of  the  DFT-of-AC  matrix. 
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Step  5  :  This  step  requires  rooting  of  the  (N  —  l)-th  order  polynomial  D(z).  Instead,  the  frequencies  may  also 
be  found  from  the  peaks  of  the  following  minimum-norm  pseudo-spectrum  [15,  26,  40]  : 


Pmnm(^u)  A 


1 

|£>(e^)|2 


(V.16) 


VI :  Perturbation  Analysis  of  the  D-MNM  Algorithm 

According  to  the  MNM  framework,  the  angles  of  arrival  0,*  are  extracted  from  the  p  roots  of  the  polynomial 
D(z)  that  are  on  or  closest  to  the  unit  circle.  Let  the  p  signal  zeros  of  D(z)  on  or  closest  to  the  unit  circle  be 
denoted  as,  Zi  —  e^Wi  which  are  found  by  rooting  D(z ). 

In  practice,  the  polynomial  .D(z),  defined  in  (V.9),  is  formed  with  the  coefficient  vector  d  estimated  using 
(V.12).  Since  d  is  a  function  of  observations,  any  error  in  d  would  be  due  to  deviations  or  noise  in  the  observa¬ 
tions.  Error  in  estimated  d  would  affect  the  estimated  roots,  Z{  and  that  in  turn  would  introduce  errors  in  the 
corresponding  u>*’s  as  well  as  in  the  s.  Hence,  in  order  to  analyze  the  bias  and  MSE  of  the  AOAs,  we  need  to 
relate  these  errors  all  the  way  back  to  the  error  in  the  MNM  coefficient  vector  d.  Hence,  we  begin  by  relating  the 
AOA  errors  to  signal  zero  errors  which  are  then  related  to  the  coefficient  errors. 


VI.  1  :  Relationships  Between  the  Errors  in  the  AOA  Estimates  and  the  Signal  Zeros 
From  (II.5)  and  the  definition  of  D(z)  in  (V.9)  we  know, 


and, 

Hence,  we  can  write, 


and, 


Similarly  for  z* , 


*  -  =  p3  — sinf 


j2*±  sin  $i 


dz{  .  2ird 

—  =  ? - cos  V;  eJ 

d$i  J  X 

A  Zi 

Adi 


A  zi  =  j  cos  2*d  sm  e'A9i . 

A 


.27 rrf 


(VI. la) 
(Vl.lb) 


(VI.  2) 


(VI. 3a) 


Az*  =  —  j -t—  cos  ^,e_J  *  sin  9'A0i . 

A 


(VIM) 

Hence,  the  bias  errors  in  AOAs  have  the  following  linear  relationship  with  the  corresponding  signal  root  errors, 


E(  A9i) 


2nd  cos  di 


»iE(Az.y 


(VIA) 


where  ’E(  )’  is  used  to  denote  the  expectation  operator.  Furthermore,  using  (VI. 3)  the  MSE  of  the  AOAs  can  be 
shown  to  be  related  to  the  signal-root  MSEs  as, 


*<1^)  = 


(VJ.5) 
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Equation  (VI. 4)  and  (VI.5)  show  that  the  bias  and  mean  squared  error  in  the  AOA  estimate  are  linear  to  those 
in  the  signal  zeros.  Next,  the  bias  and  MSE  of  signal  zeros  are  related  to  the  coefficient  errors. 


VI.2  :  Effect  of  Coefficient  Error  on  the  Bias  and  Mean-Squared  Error  of  the  Zeros 


The  effect  of  errors  in  the  coefficients  dk  on  the  zeros  of  D(z)  is  well  known  [11]  and  is  given  by  (note  that, 

do  =  1), 


dzi 

dd~k 


—z. 


-(k-i) 


k  =  1,2,..., AT  —  1. 


(VI.  6) 


Using  (VI. 6),  the  total  error  in  Z{  due  to  error  in  all  coefficients  is  given  by, 


N- 1 


A^'  *  E  trAd* 


fc=i 


ddk 


nil)*  (1  -ZIZT1) 


TV E1  zi  zl 


1  ,-2  ,-(W-2)]Ad/ 


-s/N~= T 


nil)*  (i  -^r1) 


—VH(e?Wi)Ad' 


where,  d'  is  defined  in  (V.10)  and 


V(e*w<)  A  V(*)  A  ■  . 1 . -[1  ejUi  ...  e^N~2'>Ui]T . 

-  -  yj  AT  —  1 


Hence,  the  bias  error  of  the  estimated  signal  root  is  given  by, 

-Vn^i 


E(Azi)  -  -ix 

1  li=i, (1  —  zizi  ) 


—VH(ejWi)E(  Ad') 


{VI. la) 


{VI. lb) 


(VI.  8) 


The  mean  squared  error  is  given  by, 

E(\Azi\2)  =  N~1 - —VH{e^)E{Ad’AdlH)V(e^) 

|1  ~*iZi  I2 

A  S  VH{eiUi)E{ Ad’Ad'H)V(ejUi) 


{VI. 9a) 
{VI. 9b) 


where, 


S  A 


N-  1 


=  nh)*  n-^r1!2 


{VI. 9c) 


denotes  the  sensitivity  of  the  parameter  set  and  is  a  measure  of  the  effect  of  errors  in  the  parameter  vector  d 
on  the  signal  zeros.  Equations  (VI. 8)  shows  that  the  bias  in  the  signal-zeros  is  linearly  related  to  the  bias  in 
the  coefficient  vector  and  equation  (VI. 9)  provides  the  relationship  between  the  MSE  of  signal  roots  and  the 
coefficient  error  covariance  matrix.  The  expressions  for  ^(Ad')  and  E(Ad' Ad,H)  are  derived  next. 


VI.3  :  Coefficient  Error  due  to  the  D-MNM  Method 

The  analysis  in  this  part  would  rely  on  the  assumption  made  at  the  outset;  the  observation  data  consists  of 
p  complex  sinusoids  in  additive  white  Gaussian  noise  zn(m).  Rewriting  explicitly  the  observation  matrix  defined 
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X  A  [Xi  x2  ...  xM] 

/  a?i(0)  a?2(0) 

A  *i(l)  *a(l) 

\xi(N  —  1)  x2(N-  1) 


®m(0)  \ 

xm{N  -  1)/ 


(V/.lOa) 


where, 


p 

arm(n)  A  ^ Aimeju,n  +  zm{n). 

»=i 


(V7.106) 


Note  that  the  phase  <j>m(i)  in  Aim  which  is  defined  in  (VI.6)  is  assumed  to  be  uniformly  distributed.  In  addition, 
the  theoretical  autocorrelation  matrix  C  in  (IV. 3)  can  be  written  explicitly  as, 


C  A  E(XXh) 

=  TST*  +  0*  I 

£7(*(0)**(0)) 

^(0)^(1)) 

£'(a;(0) 

**(iV  - 1))  x 

= 

£(*(1)**(0)) 

^(1)^(1)) 

...  E(x(  1) 

1)) 

[e(x(N-  1)x*(0))  E(x(N-  !)**(!)) 

...  E(x(N  — 

l)ar*(AT  —  1))  / 

C(  0)  C(- 1) 

C(- 2)  ... 

C(-(N-  1))\ 

C(l)  C(0) 

c(- 1)  ... 

e(-(Ar  -  2)) 

= 

C(  2)  C(  I) 

C(0)  ... 

C(-(7\r-3)) 

\C(N  —  1)  C(N-  2) 

C(A^  -  3)  ... 

C(0)  / 

where, 


C(m) 


E?=i  Wl?  +  <A2,  if  m  =  0; 
ELi  Wi\2ejWim,  ifm^o. 


From  Section  V.a  we  know  that  the  D-MNM  method  uses  the  following  decomposition, 


(VL 11) 


(VI.  12) 


A  CD 

(Vl.lSa) 

— >  [Fs  :  Fjv] 

(VI.  136) 

A  [fi  f2  ...  fp  :  fp+i  . . . ,  fjv] 

(VI.lZc) 

=  [Cei  Ce2  . . .  Cep  :  Cep+i  . 

. .  Cew] 

(Vl.lZd) 

=  [CDs  :  CDN ] 

(VI.  13e) 

where  p  is  the  assumed  number  of  signal  sources.  D5  contains  the  signal  subspace  DFT-vectors  e2  which  cor¬ 
respond  to  the  largest  norm  fz  vectors.  In  D-MNM,  the  vector  d  is  entirely  in  the  noise  subspace  and  must  be 
orthogonal  to  the  signal  subspace  F 5,  i.e.,  repeating  from  (V.ll) 


or,  using  (VI. 13), 

DfCd  =  0, 

(V7.146) 

Using  (V.10)  and  (V.12b), 

Gd'  =  -  g 

(VI.  15) 

and  the  error  expression  can  be  written 

as, 

A(Gd')  =  -  Ag. 

(V7.16) 

Using  chain-rule, 

AGd'  +  GAd'  =  -Ag. 

(VI.  17) 

The  pseudo-inverse  solution  for  the  coefficient  error  is, 


Ad'  =  -G#(AGd'  +  Ag) 

=  -  G#(Ag  :  AG) 

=  -G#AFfd  (VI. 18) 

where  G#  denotes  the  Moore- Penrose  pseudo-inverse  of  G.  Equation  (VI.  18)  shows  that  the  error  in  the  coefficient 
vector  depends  directly  on  the  error  of  the  signal  subspace  obtained  by  using  the  D-MNM  method.  The  coefficient 
error  in  (VI.  18)  can  now  be  utilized  in  the  expressions  for  bias  and  mean  squared  error  of  the  zeros  which  were 
derived  in  (VI. 8)  and  (VI. 9),  respectively.  However,  according  to  (VI. 8)  and  (VI. 9),  in  order  to  obtain  E(Azi) 
and  E(\Az{\2)  we  need  expressions  for  ^(A')  and  E(A'A'H),  which  are  derived  in  the  next  two  sections. 

VI.4  :  Bias  in  the  Signal  Zeros 

First  let  us  find  the  mean  of  the  coefficient  error, 

£(Ad')  =  -G#E(  AFf)d 

=  -  G#E( Df  AC*  )d;  using(V7.13) 

=  —  G#E(Ds  C)d  -f  G#D*Cd;  assumingAC  =  C  —  C 
=  -  G#Df  E(C) d  using(V/.14) 

«  —  G#D*Cd  if  covariance  is  unbiased 
=  0;  using(VI.14). 

(V7.19) 

Substituting  (VI.  19a)  in  (VI.8)  results  in, 

E(Azi)  &  0.  (V/.20) 

This  expression  implies  that  the  bias  in  the  estimate  obtained  by  using  the  D-MNM  method  can  be  expected  to 
be  quite  small.  This  fact  was  observed  in  simulations  also. 

VI. 5  :  Mean  Squared  Error  in  the  Signal  Zeros 
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For  obtaining  an  estimate  of  the  mean  squared  error  in  the  zeros,  an  expression  is  needed  for  E(Ad' Ad'n) 
which  appears  in  (VI. 9).  Starting  with  (VI. 18),  we  have, 

E(Ad'Ad,/f) 

=  G*E(AF%ddHAFs)(GH)# 

=  G*B§E(ACHddHAC)Bs(GH)*  (VI.  21) 


where, 

^(AC^dd^AC) 

=  E((CH  -  CH)ddH(C  -  C)) 

=  E(CHddHC)  -  E( CHddHC)  -  E( CHddHC)  +  E(CHddHC) 
=  E( CHddHC)  -  CHddHC 
and  from  (11.13),  we  know, 

E(  CHddHC) 


(VI.  22) 


^  M  -  M 


K  M  ^ 

*=i 


1  M  M 

E(xixflddHxjxf) 


(VI.  23) 


1=1 j= 1 


Since  this  expression  involves  fourth  moments  of  the  process,  its  computation  would  be  difficult  in  general. 
However,  for  a  Gaussian  random  process  all  higher-order  moments  can  be  expressed  in  term  of  first  and  second 
moments.  In  particular,  if  vxiV2Jv3l  and  V4  are  complex  Gaussian  random  variables,  it  is  known  that  [40], 

E(v  1V2V3VI)  =  E(viv*2)E(vzvX)  +  E(vivl)E(v3v2)  (VI.  24) 

Applying  (VI. 24)  to  (VI. 23)  leads  to  the  following  expression, 

^(C^dd^C) 


1  M  A*  j  ,  i  \  1  . 

:  (*.•(»)  •)  :  «  ■■)  :  (*;(o)...*;<w-i»} 

i=lj=1  \xi(N)J  \dNJ  \xj(N-l)J 

/  E(xi(0)x*i(k)xj(g)x*j(0))  ...  E(xi(0)x^(k)xj(g)x*j(N  - 1))  \ 

:  .  : 

ijak  \E(xi(N  -  l)x'l(k)xj(g)xj(0))  . . .  E(xf(N  -  l)x*i(k)xj(g)x*j(N  -  1)) 

(  E(xi(0)xUk))E(xj(g)x*(Q))  ...  E(xi(0)xt(k))E(xj(g)x*j(N  - 1)) 


:  (*?(o)...)  i 

%i(N) }  \dff 

E(xi(0)x:r(k)xj(g)x](0)) 


E(xi(N  -  l)x*i(k))E(xj(g)x*j(0))  . . .  E(Xi(N  -  1  )x*i(k))E(xj(g)x](N  -  1))/ 

/  E(xi  (0)a;t  (Q))E(xj  (g)x*{  (fc))  ...  E(xi(0)x*(N  -  l))E(xj(g)x*(k)) 


+  w'Ew. 

ijgk 


E(xi(N  -  l)x)(0))£(a^K(*))  •  •  •  E(Xi(N  -  l)x*(N  -  1  ))E(xj(g)x*(k))  J 

/  £(^(0)^(0))  ...  E(xi(0)x*j(N-l))  \ 

CHddHC  +  *72  Y,d^9E^3)xm)  \  :  (VI. 25) 

igk  \E(Xi(N-  1)*;(0»  ...  E(xi(N  -  l)xj(N  -  1))  / 
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where, 


M  M  N  N 

£ k ££££ 

ijgk  1  =  1  j=l  5=1  fc  =  l 


Substituting  (VI.25)  in  (VI. 22)  results  in, 
£'(ACffdd//AC) 


=  ^E^i^O/)^)) 


/  £(*,-(o)*;(o)) 


*70* 


E(ar<(0)a:?(7V  -  1)) 


U(*,(jv - i)*;(o))  ...  ^(v-i^kv-i)), 


(VJ.26) 


Further,  if  the  noise  is  white  and  the  complex  amplitudes  of  the  signal  are  uncorrelated,  then  if  i  ^  j, 
E(xj(g)x*(k))  =  0.  In  (VI. 26),  among  M2  possible  i,j  combinations,  there  are  only  M  terms  for  which,  i  =  j. 
Retaining  only  these  terms  we  have,  (assuming  stationarity,  the  dependence  on  i  (—j)  has  been  suppressed) 


E{  ACHddHAC) 

N  N 

M 


9=1 k=l 

=  c 

5=1 1=1 

=  c 


£?(*(0)**(0»  ...  E{x{0)x*{N  - 1)  \ 

E(x(N  -  1)**(0))  . . .  E{x{N  -  1  )x*{N  -  1)) ) 


{VI. 21a) 
{VI. 21b) 


{VI. 21c) 


{VI.  28) 


9=1  fc  =  l  »  =  1 

in  the  above,  6gk  is  the  Kronecker  delta  and  hence, 

E{x{g)x*{  k))  =  =  ^=1lai|2  +  ,f  9  =  k’ 

1  C(g-k)=  ELi  |aj|V"*-‘),  if 

Substituting  (VI.27b)  in  (VI.2I), 

£7(Ad'Ad'")  =  G*D§^f2f2d^(E\a<\2ejW'(9~k)  +  <r2Jgk)c)Ds{GH)*. 

5=11=1  i=l 

Using  this  in  the  expression  of  the  MSE  of  signal  zeros  in  (VI.9b), 

E{\Azi\2)  =  SVH{eiUi)G#D%(jjJ2J2dkdl(Y,\ai\2ej“'(g-k)  +  ffMc)  Ds{GHfV{e^‘)  {VI.  29) 

<7=1  fc  =  l  i= 1 

Now  we  are  ready  to  obtain  the  final  expression  for  the  bias  and  MSE  of  the  DOA  estimates. 

VI.6  :  Bias  in  the  AOA  Estimates 

Using  the  developments  in  (VI.19)-(VI.20)  into  the  AOA  bias  equation  (VI.4), 

E{A9i)  =  0.  {VI.  30) 
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VI.  7  :  Mean  Squared  Error  in  the  AO  A  Estimates 


Using  (VI.29)  in  (VI. 5),  we  finally  obtain  the  MSE  expression  of  AOAs, 


£(|A0<|2)  =  S( 


2ird  cos  6 , 


E£<K(£m 


2g  jui(g- 


■k) 


+  O’ 


0=1 k= 1 


^)c)Ds(Gff)#V(e^0 


(V/.31) 

This  expression  explicitly  shows  that  the  mean  squared  error  in  the  AOAs  not  only  depends  on  the  parameter 
sensitivities  but  also  on  the  specific  structure  in  the  D-MNM  method.  It  should  be  mentioned  here  some  of 
developments  presented  here  are  similar  to  the  work  in  [25].  However,  the  results  in  [25]  depends  heavily  on 
statistical  properties  of  eigenvalues/eigenvectors  which  are  neither  applicable  not  appropriate  in  the  present  case. 
Hence,  in  the  development  above  the  moments  of  the  data  and  covariance  have  been  used  directly.  Simulation 
studies  with  similar  data  sets  used  in  [25]  verify  close  match  between  the  theory  and  simulation  results. 


VII  :  Simulation  Results 


In  the  first  two  examples,  the  performance  of  the  proposed  D-MNM  algorithm  is  compared  with  the  perfor¬ 
mance  of  some  of  the  existing  well-known  algorithms  using  Monte-Carlo  studies.  In  Example  3,  the  theoretically 
derived  MSE  formulae  are  verified  by  comparing  theory  and  simulation  studies.  The  theoretical  performance  is 
also  compared  with  some  of  the  eigen-based  counterparts. 

VII.  a  :  DO  A  Estimation 

Simulation  1  :  Two  Closely-Spaced  Targets  of  Equal  Powers  [15,  38,  39] 

Planewaves  from  p  =  2  sources  with  =  18°  and  02  =  22°  incident  on  N=8  sensors  were  modeled  as  in  [15, 
16,  18].  The  number  of  snapshots,  M=10.  Fig.  1  shows  the  norms  of  the  f \  vectors  for  20  trials  at  20dB  SNR. 
The  two  largest  A,*s  always  appear  to  be  more  significant  than  all  the  smaller  ones.  Figures  2a  and  2b  show  the 
roots  of  D(z)  for  50  independent  realizations  using  D-MNM  and  E-MNM,  respectively.  The  figures  show  that 
the  roots  in  both  cases  are  at  almost  same  locations.  Table- 1  compares  E-MNM  and  D-MNM  in  terms  of  the 
bias  and  RMS  values  with  200  independent  trials  at  different  SNR  values.  The  results  clearly  indicate  that  the 
performance  of  D-MNM  is  quite  close  to  that  of  E-MNM,  though  no  Eigendecomposition  was  required  in  this 
case.  In  fact,  D-MNM  was  found  to  be  somewhat  more  robust  (in  terms  of  successful  trials)  at  low  SNR  ranges. 

Vll.b  :  Frequency  Estimation 

In  this  Section,  the  proposed  algorithm  is  compared  with  the  well-known  Tufts-Kumaresan  (TK)  method  [17,  42] 
and  MUSIC  method  [1,  30]  via  simulations. 

Simulation  2  :  Comparison  of  High-Resolution  Performance  and  Threshold  Enhancement 
The  simulation  data  is  generated  using  the  formula  [42]  : 

y{n)  =  aieiM°-5)n+if  +  2*(o.52)n  +  for,  n  =  0,  1,  M-  1  (VII.l) 

where,  w(n )  is  complex  white  Gaussian  noise  with  variance  <r2  .  The  number  of  data  samples  used  is,  M=25.  This 
data  set  has  been  widely  used  in  the  literature  for  studying  the  performance  of  various  methods.  For  this  data 
set,  it  has  been  shown  in  [42]  that  the  TK  method  performs  best  when  high-order  ( L  x  L)  covariance  matrix  with 
L  —  18  is  used  with  forward-backward  covariance  matrix  [42].  Five  hundred  independent  noise  realizations  were 
used  to  compare  the  performance  of  the  proposed  method  with  that  of  TK  method  and  MUSIC.  The  mean  values 
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for  three  cases  at  different  SNR  values  are  displayed  in  Fig.  4.  The  RMSE  results  are  shown  in  Fig.  5  along  with 
CR  Bound  for  the  frequency  at  f\  ~  0.52 Hz.  The  bias  and  RMSE  at  different  SNR  values  are  also  tabulated 
in  Table  2.  Clearly,  the  proposed  method  extends  the  performance  threshold  closer  to  the  CR  bound.  Hence  the 
performance  of  the  proposed  method  approaches  that  of  the  Maximum-Likelihood  method  more  closely,  although 
with  considerably  less  computational  complexity. 

VII. b  :  Perturbation  Analysis  of  D-MNM 

In  this  Section,  the  theoretical  formulae  derived  for  MSE  and  Bias  for  the  DOA  estimation  using  the  proposed 
DFT-based  algorithm  are  verified  by  comparing  theoretical  and  simulation  results. 

Simulation  3  :  Perturbation  Analysis  of  AO  As  with  Small  Number  of  Sensors 

The  problem  scenario  is  identical  to  as  described  in  Simulation  1  for  AOA  estimation.  For  this  case, 
N  =  8,M  =  100,  and  the  AO  As  are  18°  and  22°.  In  these  simulations  the  theoretical  formulas  for  mean- 
squared  error,  as  derived  in  (66)  and  (67),  are  compared  with  the  performance  using  simulated  data.  The  results 
for  various  signal-to-noise  ratios  (SNR)  are  shown  in  Fig.  6  which  shows  the  result  corresponding  to  the  AOA  of 
18°  using  all  of  the  200  independent  trials. 

Note  that  data  set  considered  in  Simulation  3  were  also  used  in  [25]  for  perturbation  analysis  of  the  eigen- 
based  MNM  and  MUSIC.  But  it  appears  that  in  [25],  only  the  results  with  successful  trials  were  plotted.  Fig. 
6  indicates  that  for  this  example,  the  D-MNM  method  appears  to  have  smaller  squared  error  than  E-MNM, 
especially  at  low  SNR.  This  was  found  to  be  the  case  for  all  trials.  In  fact,  the  success  rate  was  higher  for 
D-MNM  when  compared  with  E-MNM.  Following  the  trend  in  the  eigen-based  cases,  Root-MUSIC  fared  a  little 
better  (except  at  low  SNR),  but  it  may  be  noted  that  in  case  of  Root-MUSIC  the  polynomial  to  be  rooted 
has  twice  the  order  than  either  D-MNM  or  E-MNM.  Finally,  it  may  be  emphasized  here  that  the  theoretical 
predictions  based  on  formulas  derived  in  this  paper  were  found  to  be  quite  close  to  those  obtained  by  computer 
simulations. 

VIII  :  Analysis,  Discussion  and  Directions  on  Further  Research 

The  results  presented  so  far  are  quite  intriguing  and  can  may  possibly  have  some  important  consequences  on 
simplifying  the  present  practice  of  frequency/DOA  estimation.  The  proposed  approach  of  forming  signal-subspace 
using  DFT  without  any  eigendecomposition  also  opens  up  whole  new  avenues  for  further  research  and,  at  the 
same  time,  poses  some  unanswered  questions.  Furthermore,  it  may  be  possible  to  extend  and  incorporate  similar 
ideas  in  other  closely  related  problems  or  to  develop  more  simplified  algorithms.  Clearly,  the  major  advantage  of 
the  proposed  approach  is  that  all  the  signal-subspaces  are  obtained  with  a  single  matrix  multiplication.  This  step 
may  be  performed  using  FFT  which  is  very  efficient  for  hardware  and  software  implementation.  In  the  following, 
some  analysis  as  well  as  some  possible  directions  for  further  research  are  briefly  discussed. 

1.  Reduced  Computational  Complexity  and  Usefulness  in  High  Sampling- Rate  Problems  :  The 

major  significance  of  D-MNM  is  that  its  high-resolution  capability  does  not  rely  on  any  iterative  method 
or  eigendecomposition  which  is  also  computed  iteratively.  The  lower  computational  complexity  of  D-MNM 
should  be  attractive  in  any  general  frequency/DOA  estimation  scenario.  But  the  usefulness  of  the  proposed 
method  should  be  specially  significant  in  those  applications  where  traditional  high-resolution  methods  are  yet 
to  make  much  inroads  due  mainly  to  extremely  high  sampling  rate  requirements.  Specifically,  in  Electronic 


29 


Warfare  (EW)  applications,  the  signals  usually  operate  in  the  GHz  range  but  real-time,  high-resolution 
capability  is  a  necessity  [41].  Currently  no  EW  receiver  processes  signals  entirely  in  digital.  The  proposed 
DFT-based  MNM  with  its  low  computational  complexity,  is  expected  to  provide  the  desired  high-resolution 
capability  to  future  digital  EW  receivers. 


2.  Signal- Subspace  Information  from  the  Autocorrelation  Matrix  Only  :  The  strength  of  the 
Minimum-Norm  framework  as  a  high-resolution  method  really  comes  from  its  ability  to  form  the  ‘noise- 
subspace’  vector  d  by  exploiting  the  orthogonality  property  in  (VT1).  It  appears  that  as  long  as  F5  has 
some  component  of  the  signal-subspace  T,  the  solution  of  (Kll)  would  retain  its  high-resolution  capabil¬ 
ity.  The  DFT-of-AC  is  an  appropriate  candidate  to  produce  F5  because  it  is  a  linear  combination  of  the 
signal- vectors  in  T.  This  can  be  seen  by  rewriting  the  DFT-of-AC  matrix, 


(VIII.  1) 

In  fact,  the  AC  matrix  itself  is  also  a  possible  candidate  for  obtaining  the  ‘signal-subspace’  F 5,  because  it 
can  be  expressed  as  a  linear  combination  of  the  signal- vectors  in  T, 


F  =  CD  =  T 


1  M 


(xmD) 


C  =  T 


1  M 


amxm 


m=l 


(VIII.2) 


Theoretically,  the  norm  of  each  vector  in  ideal  C  should  be  equal  but  with  noisy  C,  the  norms  of  some  of 
the  vectors  may  be  reduced  while  for  other  vectors,  the  norms  may  be  more  than  the  nominal  value.  Hence, 
the  ideal  choice  would  be  to  pick  the  vectors  with  norms  in  the  middle  range.  Not  surprisingly,  when  Fs  is 
formed  in  this  manner  with  p  vectors  of  the  estimated  C,  MNM  again  demonstrated  high-resolution  capability 
in  simulations  (not  included).  This  simpler  procedure  to  obtain  ‘signal-subspace’  information  needs  to  be 
studied  further.  But  it  must  be  stated  that  D-MNM  performs  better  at  low  SNR  because  the  DFT  operation 
accentuates  the  signal-subspace,  as  discussed  next. 

3.  Asymptotic  Analysis  of  the  DFT-based  Signal  Subspace  for  Arbitrary  DO  A/Frequency  :  For 

ideal  noise-free  observations  if  the  frequencies  are  not  on  the  DFT  bins,  the  DFT-of-AC  operation  can  be 
expressed  as  : 


F  =  CD 

(VUI.ia) 

=  TST^D 

(VIIIM) 

=  TE 

rtfDi 

t?D 

(V  111.3c) 

-t?D. 

Consider  the  matrix  at  right.  Each  of  the  t-*D  vectors  are  complex  valued  DFT  of  a  sequence  of  a  complex 
sinusoid.  The  magnitude  of  each  row  vector,  t^D  has  a  Sine  envelope  with  a  peak  occurring  at  the  column 
corresponding  to  the  bin  location  closest  to  the  frequency  c Jj.  For  infinite  aperture  with  N  — ►  00,  i.e.,  for 
large  number  of  sensors,  each  row  vector  peaks  at  u and  the  other  elements  of  that  row  approaches  zero. 
The  same  will  be  the  case  for  each  of  the  other  row  vectors  also.  Hence,  asymptotically,  the  DFT-of-AC 
operation  again  produces  p  largest  norm  vectors  at  the  true  signal  frequencies.  The  asymptotic  analysis 
for  the  noisy  case  as  defined  by  (V.7)  would  also  provide  similar  results.  For  finite  Nt  because  of  the  Sine 
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weighting,  the  largest  norm  vectors  will  also  have  contributions  from  some  other  t*  vectors  in  the  T.  But 
those  components  also  contain  signal-subspace  information  which  is  orthogonal  to  d  and  hence  useful  for 
obtaining  the  minimum-norm  vector  d. 

4.  Estimation  of  the  Parameters  of  Damped  Sinusoids  in  Noise  :  Many  eigen-based  methods  have  been 
successfully  utilized  in  estimating  the  unknown  parameters  of  damped  sinusoids  from  noisy  observations  [5, 
14].  It  appears  that  with  some  simple  modifications  the  proposed  DFT-based  approach  could  also  be  used 
for  the  same  purpose.  The  advantage  would  again  be  that  no  eigendecomposition  but  the  performance  will 
be  comparable. 

5.  Largest  Norms  vs.  Peaks  :  In  all  the  simulations  presented  here,  the  signal  subspaces  have  been  formed 
by  selecting  the  p  unit-vectors  having  largest  norms.  But  the  ideal  solution  may  be  to  pick  the  unit  vectors 
corresponding  to  the  p  largest  peaks  (having  smaller  norm  vectors  on  both  adjacent  bins).  This  may  eliminate 
any  possibility  of  picking  multiple  vectors  from  the  vicinity  of  strong  signals.  It  should  be  emphasized  though 
that  largest  norm  criteria  has  worked  quite  well  so  far,  as  demonstrated  by  a  large  number  of  simulations. 
But  this  aspect  certainly  needs  further  analysis. 

6.  Zero  padding  :  In  classical  spectral  estimation,  Periodogram  relies  on  DFT/FFT,  but  it  is  often  necessary 
to  extend  (or,  pad)  the  available  data  with  zeros  so  that  interpolated  values  between  available  bins  can  be 
calculated.  Zero-padding  is  also  used  to  extend  data-lengths  to  powers  of  two  such  that  the  computational 
efficiency  of  the  FFT  can  be  taken  advantage  of.  In  the  simulation  studies,  no  zero-padding  had  been 
incorporated  so  far.  It  is  not  quite  apparent  whether  the  zero-padding  should  be  done  directly  to  the  data 
or  to  the  covariance  estimates  and  this  aspect  needs  further  study.  It  would  also  be  necessary  to  study  the 
possible  effects  on  the  signal-subspace  produced  by  the  DFT-of-AC  operation  after  zero-padding  is  introduced. 

7.  Windowing  :  In  classical  spectral  estimation,  in  order  to  avoid  sudden  discontinuities,  the  observed  data  is 
often  weighted  (or  tapered  at  both  ends)  by  non-rectangular  window  which  tends  to  enhance  the  'dynamic 
range’  at  the  cost  of  ‘resolution’.  In  the  simulation  results  presented  here,  no  windowing  has  been  used. 
But  windowing  is  known  to  be  highly  effective  in  locating  weak  frequency  components  which  tend  to  get 
submerged  by  the  sidelobes  of  strong  components.  Though  it  is  believed  that  that  orthogonality  property 
in  (VT1)  is  the  main  contributing  factor  for  the  high-resolution  capability  of  D-MNM,  it  would  certainly  be 
interesting  to  study  what  effects  windowing  might  have  on  the  performance  of  D-MNM. 

8.  Use  of  DFT-Based  Signal- Subspace  in  other  Eigen-Based  Methods  :  Other  than  the  Minimum- 
Norm  Method  covered  in  this  paper,  there  is  a  large  body  of  work  where  some  form  of  eigendecomposition 
is  utilized  to  estimate  DOA/Frequencies  [1,  2,  8,  10,  13-15,  19,  22,  23,  30,  32,  36,  42,  43,  44-47,  49].  Among 
the  more  important  results  are,  MUSIC  [30],  SVD  [15,  42]  and  ESPRIT  [22].  It  is  quite  possible  that  the 
proposed  DFT-based  signal-subspace  may  be  incorporated  with  some  of  these  existing  eigendecomposition 
based  methods,  in  order  to  implement  those  methods  without  eigendecomposition.  Clearly,  the  proposed 
approach  can  be  used  to  implement  MUSIC,  except  that  the  noise  subspace  F n  defined  in  (U.8c)  would  have 
to  be  utilized.  Also,  the  left  and  right  eigenvectors  of  the  SVD  of  a  data  matrix  are  actually  the  eigenvectors 
of  correlation  matrices.  Hence,  it  appears  that  some  of  the  SVD-based  approaches  may  also  be  modified 
to  incorporate  DFT-based  signal/noise  subspaces.  Care  should  be  taken  about  the  choice  of  either  the  left 
or  right  signal-spaces,  because  both  may  not  contain  signal  information.  The  case  is  not  so  apparent  for 
those  methods  which  use  generalized  eigendecomposition  [22,  36,  45].  As  part  of  this  paper,  some  of  these 
possibilities  will  be  further  investigated. 
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9.  DFT-Prony  :  There  has  been  some  recent  interest  in  implementing  the  Prony’s  algorithm  in  the  Frequency- 
Domain  [29].  Clearly,  the  signal- vectors  in  F  5  can  be  treated  as  multiple  time-series  to  form  a(p+l)x(p+l) 
covariance  matrix  (using  forward-backward  approach)  and  then  the  p-th  order  Prony’s  polynomial  can  be 
estimated.  Based  on  preliminary  simulations  (not  included),  this  approach  appears  to  be  simplest  of  all 
existing  methods  with  moderately  good  high-resolution  performance.  The  performance  of  DFT-Prony  is 
much  better  than  that  of  the  standard  Prony’s  method  because  the  DFT-based  signal  subspace  is  cleaned-up 
though  without  any  eigendecomposition.  These  ideas  will  be  further  studied  as  part  of  this  paper. 

10.  Two-Dimensional  Frequency- Wavenumber  Estimation  :  In  some  array  processing  scenarios,  both 
the  DOAs  (related  to  wavenumbers)  and  the  center  frequencies  need  to  be  estimated.  Many  existing  1-D 
eigen-based  methods  have  been  extended  to  2-D  to  address  this  problem.  It  appears  that  the  DFT-of-AC 
vectors  can  be  formed  in  both  domains  and  two  D(z)  polynomials  can  be  be  formed  to  estimate  the  the 
frequencies  and  DOAs  separately.  Incorporation  of  the  DFT-based  signal-spaces  for  2D  frequency  estimation 
will  be  further  investigated  as  part  of  this  paper. 

11.  Hardware  Implementation  :  Perhaps  the  most  important  and  useful  practical  impact  of  the  proposed 
method  would  be  in  the  area  of  hardware  implementation  for  high-resolution  Direction-of- Arrival  or  frequency 
estimation.  All  the  currently  available  methods  with  good-enough  high-resolution  capability,  rely  on  some 
form  of  iterative  optimization  or  iterative  computation  of  eigenvectors.  In  contrast,  all  that  the  proposed 
approach  requires  to  form  the  ‘signal-subspace’  is  a  single  matrix  multiplication.  Furthermore,  the  matrix  to 
be  multiplied  is  a  DFT  matrix  and  it  has  special  structures  so  that  FFT  based  processing  may  be  utilized  to 
further  reduce  the  computational  burden.  Hence,  one  of  the  major  goals  of  the  proposed  work  would  be  to 
devise  appropriate  strategies  to  design,  develop  and,  if  possible,  fabricate  VLSI  hardware  for  high-resolution 
DOA/Frequency  estimation. 


Fig.l.  Norms  of  the  DFT-of-AC  vectors 
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Comparison  of  Performance 


FigS  Performance  comparision  of  three  methods 
using  all  200  trials. 


Fig-2.  Roots  of  D(z)  using  (a)  E-MNM  and  (b)  D-MNM  for  50  independent 
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Fig.4.  Compaxison  of  Mean  values  with  500  independent 
trials  for  three  methods. 
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Fig.5.  Comparison  of  RMS  values  with  CR  bounds  for  500 
independent  trials. 
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SNR 

Successful  Trials 

r~  RMS 

(in  dB) 

D-MNM 

E-MNM 

D-MNM 

E-MNM 

D-MNM 

E-MNM 

5 

59 

39 

-0.8480 

1.1589 

-0.5539 

0.4329 

1.4311 

1.9174 

1.3623 

1.9322 

10 

139 

130 

-0.3154 

0.8940 

-0.4589 

0.7603 

1.3529 

1.7910 

1.5063 

1.8571 

15 

191 

189 

-0.0714 

0.4623 

-0.1094 

0.3648 

0.9812 

1.3118 

1.0021 

1.3212 

20 

199 

198 

-0.0055 

0.1717 

-0.0252 

0.1170 

0.6777 

0.8440 

0.6822 

0.8017 

25 

'  200 

200 

4.99e-4 

0.0611 

-0.0067 

0.0481 

0.4129 

0.4820 

0.4302 

0.4826 

30 

200 

’200 

0.0037 

0.0263 

0.0018 

^0.0219 

0.2297 

0.2728 

0.2329 

0.2737 

Table  1  :  Comparison  of  performance  of  D-MNM  and  E- 
MNM. 


SNR 

Bias  (in  degrees) 

RMS 

(in  dB) 

D-MNM 

E-MNM 

MUSIC 

D-MNM 

E-MNM 

MUSIC 

0 

-0.0349 

0.0352 

-0.1205 

0.1118 

-0.1178 

0.0983 

0.0876 

0.0786 

0.1783 

0.1748 

0.1594 

0.1486 

3 

-0.0133 

0.0141 

-0.1029 

0.1027 

-0.0681 

0.0678 

0.0415 

0.0468 

0.1654 

0.1640 

0.1312 

0.1265 

5 

-0.0070 

0.0072 

-0.0964 

0.0838 

-0.0343 

0.0392 

0.0232 

0.0342 

0.1476 

0,1378 

0.0946 

0.0991 

7 

-0.0031 

0.0039 

-0.0754 

0.0658 

-0.0063 

0.0111 

0.0142 

0.0245 

0.1322 

0.1189 

0.0373 

0.0560 

10 

-3.40e-4 

6.54e-4 

-.0.0289 

0.0301 

-5.62e-4 

-1.19e-4 

0.0054 

0.0093 

0.0756 

0.0776 

0.0140 

0.0058 

15 

-7.10e-5 

-1.82e-4 

-0.0023 

0.0019 

2.80e-5 

-9.05e-5 

0.0020 

0.0022 

0.0159 

0.0134 

0.0026 

0.0026 

20 

-3.40e-6 

-7.76e-5 

-1.04e-5 

6.34e-5 

1.61e-5 

-5.23e-5 

0.0011 

0.0012 

0.0025 

0.0025 

0.0015 

0.0014 

30 

8.64e-6 

-1.77e-5 

2.18e-5 

-9.01e-6 

3.35e-6 

-1.51e-5 

3.53e-4 

3.75e-4 

7.87e-4 

7.84e-4 

4.61e-4 

4.50e-4 

Table  2.  Comparison  of  Bias  and  RMS  values  for  three  methods 
with  500  independent  trials. 
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Section  -  2.2  :  MAXIMUM-LIKELIHOOD  ESTIMATION  OF  MULTIPLE  FREQUENCIES  WITH 

Constraints  to  Guarantee  Unit  Circle  Roots 
Summary 

A  recently  proposed  approximate  Maximum- Likelihood  Estimator  (MLE)  of  multiple  exponentials,  converts 
the  frequency  estimation  problem  into  a  problem  of  estimating  the  coefficients  of  a  z-polynomial  with  roots  at 
the  desired  frequencies  [1,  2].  Theoretically,  the  roots  of  the  estimated  polynomial  should  fall  on  the  unit  circle. 
But  MLE,  as  originally  proposed,  does  not  guarantee  unit  circle  roots.  This  drawback  sometimes  causes  merged 
frequency  estimates,  especially  at  low  SNR  [1,  3].  If  all  the  sufficient  conditions  for  the  z-polynomial  to  have 
unit  circle  roots  are  incorporated,  the  optimization  problem  becomes  too  nonlinear  and  it  loses  the  desirable 
weighted-quadratic  structure  of  MLE.  In  this  paper,  the  exact  constraints  are  imposed  on  each  of  the  lst-order 
factors  corresponding  to  individual  frequencies  for  ensuring  unit  circle  roots.  The  constraints  are  applied  during 
optimization  alternately  for  each  frequency.  In  the  absence  of  any  merged  frequency  estimates,  the  RMS  values 
more  closely  approach  the  theoretical  Cramer- Rao  (CR)  bound  at  low  SNR  levels. 


I.  Introduction 

Estimating  the  underlying  parameters  of  multiple  complex  exponential  signals  in  noise  remains  a  vigorously 
researched  topic  in  signal  processing  [1-13].  For  a  single  sinusoid  or  when  the  multiple  frequencies  are  well- 
separated,  the  Periodogram  performs  reasonably  well.  But  if  the  frequencies  are  closely  spaced,  which  often  occurs 
when  the  data  length  is  limited  or  the  aperture  is  too  small,  the  Periodogram  fails  to  distinguish  the  frequencies 
and  produces  merged  frequency  estimates.  In  order  to  overcome  the  Periodogram’s  resolution  limitation,  many 
high-resolution  methods  have  been  developed  in  the  past  two  decades  [1-13].  In  contrast  to  the  Periodogram, 
these  methods  make  effective  use  of  some  underlying  property  of  the  true  sinusoidal  signal  model. 

Among  all  the  existing  high- resolution  frequency  estimation  methods,  the  MLE  appears  to  provide  the  most 
accurate  frequency  estimates  and  has  the  lowest  SNR  threshold  [1-4].  Other  high-resolution  methods  rely  on 
signal  or  noise  subspace  information  which  is  extracted  from  the  eigendecomposition  of  covariance  matrix  or  SVD 
of  data  matrix  [5,  7-11].  On  the  other  hand,  the  MLE  considers  the  exact  model  of  the  exponential  signal  and 
attempts  to  maximize  the  exact  likelihood  function  to  estimate  the  unknowns.  For  a  single  sinusoid,  the  peak  of 
the  periodogram  itself  corresponds  to  the  ML  estimate,  but  for  multiple  exponentials  the  MLE  turns  out  to  be  a 
nonlinear  optimization  problem  [1-6,  12,  13]. 

The  MLE  approaches  developed  independently  in  [1]  and  [2],  estimate  the  frequencies  from  the  roots  of  a 
z-polynomial.  It  may  be  noted  here  that  in  literature,  these  methods  are  sometimes  referred  to  as  KiSS  [1,5, 
6]  or  IQML  [2].  In  the  polynomial  domain,  the  ML  optimization  problem  turns  out  to  be  quasi-linear  where  a 
weighted-quadratic  criterion  is  minimized  iteratively.  Though  effective  to  a  large  extent,  MLE  is  known  to  possess 
one  fundamental  drawback  :  the  optimization  procedure  in  [1,  2]  does  not  impose  sufficient  theoretical  constraints 
on  the  polynomial  coefficients  for  the  estimated  roots  to  fall  on  the  unit  circle.  The  primary  goal  of  this  work  is 
to  address  this  unresolved  problem  in  MLE. 

Two  conditions  must  be  satisfied  for  a  general  p-th  order  z-polynomial  to  have  p  unit  circle  roots  :  conjugate 
symmetry  (Cl)  and  a  derivative  constraint  (C2),  the  details  of  which  are  given  later.  In  MLE,  only  Cl  is 
imposed.  The  derivative  constraint  makes  the  problem  highly  nonlinear  and  hence,  C2  can  not  be  incorporated 
in  the  weighted-quadratic  framework  of  MLE.  But  when  p  >  1,  Cl  alone  is  not  sufficient  for  unit  circle  roots. 
Furthermore,  from  the  theory  of  Linear-Phase  FIR  filters,  it  is  well-known  that  the  roots  of  a  symmetric  z- 
polynomial  may  fall  either  on  the  unit  circle  or  they  may  be  in  reciprocal  pairs  falling  inside  and  outside  of 
the  unit  circle.  In  fact,  it  was  demonstrated  in  [1]  and  [3]  that,  if  SNR  <  lOdB  and  the  frequencies  are  spaced 
closely,  the  roots  extracted  by  MLE  sometimes  appear  in  reciprocal  pairs.  In  such  cases,  two  frequencies  merge 
to  produce  only  a  single  frequency  estimate.  The  alternate  approach  proposed  in  this  paper  attempts  to  alleviate 
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this  limitation. 

There  is  one  exception  to  the  two  conditions  stated  above  :  for  p  —  1,  the  conjugate  symmetry  constraint 
(Cl)  alone  is  sufficient  for  the  single  root  to  fall  on  the  unit  circle.  This  is  the  main  idea  which  will  be  utilized 
in  developing  the  proposed  Constrained-MLE  (C-MLE)  algorithm.  Specifically,  Cl  will  be  imposed  on  each  of 
the  lst-order  factors  of  the  p-th  order  z— polynomial,  such  that  each  individual  root  falls  on  the  unit  circle.  This 
process  need  not  be  applied  to  all  the  frequencies  at  all  SNRs.  The  constraints  are  imposed  only  on  those  lst-order 
factors  which  produce  merged  frequency  estimates  at  convergence  of  MLE.  The  factors  for  which  the  roots  are 
already  on  the  unit  circle,  are  held  fixed.  The  proposed  algorithm  may  be  considered  to  be  a  polynomial-domain 
counterpart  of  the  ‘Alternating  Projection’  approach  [13]  where  the  ML  criterion  is  minimized  w.r.t.  one  frequency 
at  a  time  while  the  other  frequencies  are  held  at  the  previously  estimated  values.  Our  work  appears  to  be  the 
first  attempt  to  guarantee  unit  circle  roots  on  the  polynomial  coefficients  for  Maximum-Likelihood  frequency 
estimation.  The  constraints  are  primarily  effective  at  low  SNR  levels  when  there  is  a  higher  possibility  for  MLE 
to  produce  merged  frequency  estimates.  In  simulations,  the  RMS  values  of  the  frequency  estimates  using  C-MLE 
were  found  to  be  closer  to  the  theoretical  CR  bounds  than  those  of  the  original  MLE  algorithm. 

The  paper  is  arranged  as  follows  :  In  Section-II,  the  ML  problem  is  stated  and  the  original  MLE  algorithm  is 
briefly  discussed  and  the  conditions  needed  for  unit  circle  roots  are  stated.  In  Section  III,  the  proposed  constrained 
version  of  MLE  is  introduced.  Simulation  results  are  given  in  Section-IV  to  verify  the  performance  of  C-MLE. 


II.  The  Maximum  Likelihood  Problem  and  a  Brief  Overview  of  MLE 

The  observed  samples  of  a  complex  multiple  exponential  signal  can  be  represented  as 

p 

x(n)  A  J2  +  z(n )  n  =  0,1,...,  A-  1, 


where,  u;*,  Ck  and  <j>k  are  the  unknown  angular  frequency,  amplitude  and  phase,  respectively,  of  the  kih  sinusoid; 
p  is  the  assumed  number  of  sinusoids  and  z(n)  represents  i.i.d.  Nft^cr2)  Gaussian  noise  samples.  For  this  signal 
model,  the  MLE  corresponds  to  optimization  of  the  following  error  criterion  [1-4]  : 


where, 


min  ||e||2  A  min  ilx 

wi)...,wJ)|flil...,Op  =  wi1...,wpiail...1flp 


A  Ta 


x(N  -  1) 


0ju>i(N—l)  jw2(N-l) 


pjup(N-l) 


ah  A  Cke^k ,  for  k  —  l,2,...,p,  respectively,  are  the  complex  amplitudes.  The  MLE  problem  stated  in  (2)  is 
a  nonlinear  optimization  problem  with  respect  to  the  angular  frequencies.  Instead,  MLE  forms  an  alternative 
but  equivalent  error  criterion  in  the  polynomial  coefficient  domain  which  has  a  quasi-linear  structure  which  is 
well-suited  for  iterative  optimization.  A  brief  summary  of  the  MLE  criterion  is  in  order. 

Let,  B(z)  A  60  +  &i z~l  -f  ...  +  bpz~p}  be  a  pth  degree  z-polynomial  with  p  roots  at  eJWl,  e^2  . . . 

eja;p,  respectively,  and  b  A  [bo  6X  bp]T  be  the  coefficient  vector.  The  MLE  criterion  for  estimating  b  is 

[l]-[4]  : 

min  £(b)  =  bHXH (BBff)“1Xb  where,  (4) 
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(5) 


The  criterion  in  (4)  appears  to  be  quadratic  in  b,  except  that  the  weight  matrix  itself  depends  on  the  unknown 
coefficients.  Hence,  this  criterion  is  minimized  iteratively.  At  the  ( k  —  l)-th  iteration 


min 

b 


(6) 


is  optimized,  where  the  weight  matrix  (BBH)  is  formed  using  the  estimate  of  b  found  at  the  previous  iteration. 
At  convergence  of  these  iterations,  the  frequencies  are  found  from  the  roots  of  the  estimated  polynomial  B(z). 
Unfortunately,  direct  optimization  of  the  criterion  in  (4)  does  not  guarantee  that  the  roots  of  B(z )  will  indeed 
fall  on  the  unit  circle  and  it  was  recognized  in  [1,  3]  that  two  conditions,  must  be  satisfied  to  guarantee  unit  circle 
roots  : 


Cl  :  The  coefficients  possess  conjugate  symmetry  : 


bk  =  b*_ for,  fc  =  0,1  and, 

C2  :  For  p  >  1,  the  derivative  of  B(z ),  i.e.y 


(7) 


B’(z) 


dB{z_ 

dz~ 


(8) 


must  have  zeros  either  inside  or  on  the  unit  circle. 

The  polynomial  domain  MLE,  as  originally  proposed,  imposes  the  conjugate  symmetry  constraint  only  [1,2]. 
C2  makes  the  optimization  problem  highly  nonlinear  and  the  weighted-quadratic  structure  of  (4)  is  lost  if  C2  is 
incorporated  in  the  algorithm.  Hence,  no  attempt  was  made  in  [1-4]  to  include  C2  in  the  algorithm.  But  if  p  >  1, 
Cl  is  not  a  sufficient  condition  for  unit  circle  roots.  The  same  condition  may,  in  fact,  lead  to  roots  in  reciprocal 
pairs  which  can  and  does  occur  in  MLE,  especially  at  low  SNR.  In  such  cases,  two  closely  spaced  frequencies  are 
estimated  as  a  single  frequency  only  [1,  3]. 


Important  Observation  :  For  p  =  1,  the  conjugate  symmetry  alone  is  a  sufficient  condition  to  ensure  unit- 
circle  root.  Hence,  we  propose  to  impose  Cl  sequentially  on  each  lst-order  factor  of  B(z)  during  optimization  of 
(4).  In  that  case,  the  optimization  at  each  step  will  be  with  respect  to  only  a  lst-order  factor  of  B(z)  and  hence, 
there  is  no  need  for  satisfying  C2. 


III.  Constrained  MLE  (C-MLE) 

The  p-th  order  polynomial  B{z)  can  be  expressed  in  factored  form  as  : 

B(z)  =  B^~i\z)B^\z),  (9) 

where,  B(p~'\z)  A  6qP_^  +  +  ...  +  b^S^z~p+1  and  B^\z)  A  6^  +  b^z~x,  are  (p  —  1)- 

th  order  and  Ist-orcTer  factors,  respectively.  If  conjugate  symmetry  is  imposecTon  the  1st  order  factor,  then, 
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B^\z)  —  6g  ^  +  b^z  1.  Note  that  in  (9)  the  coefficients  of  the  polynomial  B(z)  are  formed  as  the  convolution 
of  the  coefficients  of  B^p~%\z)  and  B^\z).  Hence,  in  matrix-vector  notation  : 


Mp_0  o 

h{p-i)  h(v-i) 

b  =  :  : 

iip- 0  liv- 0 

°p- 1  °p- 2 

\  0  ^_7> 

1E--«(|  Lj)  (j|)  AB,_iCb,, 


(10) 


where,  B denotes  the  matrix-factor  with  the  i-th  1-st  order  factor  removed  and  A  b$  +  jb^ .  Using 
(10)  in  (6),  each  Ist-order  factor  of  B(z)  is  estimated  by  optimizing, 


min  bi[CHB*i(k  1]XH l))-1XBfc1)C]bi> 


hi 


for,  i  =  1,2 

This  is  a  weighted-quadratic  criterion  of  the  form  : 

where, 


(11) 
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bi  A 


(126) 


Note  that  the  weight  matrix  is  formed  with  the  estimates  found  at  (k  —  1)  —  th  iteration  step  when  the 

unconstrained  MLE  algorithm  is  assumed  to  have  converged.  The  criterion  in  (11)  can  be  optimized  sequentially 
or  concurrently  for  each  first  order  factor.  At  each  iteration,  b,  is  estimated  as  the  eigenvector  corresponding 
to  the  minimum  eigenvalue  of  £  Ht2x2.  The  advantage  of  using  (12a)  instead  of  (6)  is  that,  since  each 

B&(z)  is  a  first-order  ^-polynomial,  the  conjugate  symmetry  constraint  is  sufficient  to  guarantee  the  root  of 
B^l\z)  to  fall  on  the  unit  circle.  In  practice,  the  alternate  optimization  procedure  in  (11)  need  not  be  carried  out 
for  all  the  p  factors  of  B(z).  It  needs  to  be  invoked  only  in  those  cases  for  which  unconstrained  MLE  produces 
merged  frequency  estimates.  The  roots  which  are  already  on  the  unit  circle  need  not  be  optimized  further.  This 
sequential  process  guarantees  that  all  the  roots  of  B(z)  will  indeed  fall  on  the  unit  circle. 


IV.  Simulation  Results 

The  algorithm  described  in  this  paper  has  been  tested  with  the  same  simulated  data  set  used  in  [1]  and  [2],  The 
following  formula  was  used  to  generate  the  data, 

x{n)  =  aie^in  +  a2ej^n  +  z(n)  (13) 

n  =  0,  1,  ...,24 

where,  u)\  —  27t/i,  uj2  =  27r/2,  f\  and  f2  being  0.52  and  0.50,  respectively,  a\  =  1,  a2  =  ^ ,  z(n)  is  a  computer 

generated  white  zero- mean,  complex  gaussianly  distributed  noise  sequence  with  variance  =  cr2,  z.e.,  ~  is  the 
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variance  of  the  real  and  the  imaginary  parts  of  z(n).  SNR  is  defined  as,  10  log10(^^-).  Two  hundred  data  sets 
with  independent  noise  epochs  were  used. 

Fig.  la  and  lb  show  the  estimated  roots  for  200  independent  trials  of  MLE  for  SNR  =  5dB  and  lOdB, 
respectively.  Fig.  Id  and  le  show  the  corresponding  results  with  C-MLE.  For  the  lOdB  case,  Figures  lc  and  If 
show  only  the  merged  cases  before  after  applying  the  exact  constraints.  The  unit  circle  roots  in  Fig.  If  does  show 
wider  spread  than  the  corresponding  merged  frequency  estimates  in  Fig.  lc.  Fig.  2  compares  the  performance  of 
MLE  and  C-MLE  with  the  theoretical  CR  bound.  The  results  verify  that  C-MLE  performs  better  than  original 
MLE  at  low  SNR  range.  The  performance  of  C-MLE  has  also  been  compared  with  that  of  the  AP  method  [13]  and 
the  results  are  displayed  in  Fig.  3.  Clearly,  the  proposed  method  outperforms  the  AP  method  for  this  example, 
especially  at  low  SNR. 
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ESTIMATES  USING  MLE  ■  ESTIMATES  USING  C-MLE 


Fig.  1  .  Superimposed  plots  of  estimated  roots  for  200  independent  trials  using  MLE  (a-c)  and 
C-MLE  (d-f). 


Performance  comparison  of  C-MLE  and  AP  methods  with  the  theoretical  CR-bound. 
200  independent  trials  were  used. 


Section  -  2.3  :  IMPROVED  AR-PARAMETER  ESTIMATION  FROM  NOISY  OBSERVATION  DATA 

Summary 


Auto- Regressive  (AR)  modeling  is  the  most  widely  used  approach  for  model-based  spectrum  estimation. 
But  almost  all  the  existing  methods  for  AR-parameter  estimation  show  severe  degradation  if  the  observed  signal 
is  corrupted  with  noise.  In  fact,  all  the  commonly  used  techniques,  such  as,  Autocorrelation  Method  (AM), 
Covariance  Method  (CM),  Modified  Covariance  Method  and  their  variations,  give  poor  Power  Spectral  Density 
(PSD)  estimates  when  the  observations  are  noisy.  In  this  Section,  a  data- adaptive  pre-filtering  approach  is 
presented  to  address  this  problem.  Preliminary  results  indicate  that  when  only  noisy  data  is  available  for  modeling, 
the  proposed  technique  gives  more  accurate  PSD  estimates  than  the  commonly  used  methods. 


Introduction 

Auto-regressive  (AR)  modeling  continues  to  play  a  very  important  role  in  model-based  spectral  estimation 
[1-4].  A  major  reason  for  the  wide  appeal  of  Auto-regressive  (AR)  modeling  is  its  computational  simplicity. 
Specifically,  the  standard  AR-methods  such  as,  Covariance  method  or  Autocorrelation  method  or  their  variations 
only  need  to  solve  a  set  of  linear  equations.  Furthermore,  in  estimating  ARMA  or  MA  models,  AR-parameter 
estimation  is  a  necessary  intermediate  step  [1].  But  there  remains  a  fundamental  problem  with  most  AR- modeling 
methods  and  that  is  with  regards  to  the  sensitivity  of  the  AR  spectral  estimators  to  observation  noise.  Noisy 
observation  samples  are  indeed  very  common  in  practice,  and  the  performance  of  the  existing  estimators  deteri¬ 
orate  drastically  in  such  cases.  There  have  been  some  previous  attempts  to  address  this  problem.  AR-model  in 
noise  being  a  special  type  of  ARMA  model,  this  property  has  been  used  in  [8,  9],  but  this  makes  the  estimation 
problem  highly  nonlinear.  Another  suggested  solution  has  been  to  model  the  process  as  large-order  AR  model 
so  as  to  reduce  the  estimation  bias,  but  this  may  lead  to  spurious  peaks  if  the  chosen  model  order  is  too  high 
[11],  Other  methods  suggest  noise  compensation  to  remove  the  bias  but  this  requires  prior  information  about  the 
observation  noise  [10]. 

The  main  goal  of  this  Section  is  to  utilize  certain  data-prefiltering  ideas  which  have  been  found  to  be  highly 
effective  in  estimating  sinusoidal  frequencies  from  noisy  data  [5,  6]  and  also  for  identifying  deterministic  systems 
from  Input-Output  data  [12]  and  Impulse  Response  Data  [13].  It  is  well-known  that  a  sinusoidal  process  can  be 
viewed  as  a  limiting  case  of  a  narrowband  AR-process.  Indeed,  the  peak  locations  of  AR-spectra  are  commonly 
used  as  the  estimates  of  frequencies  [1].  But  the  poor  performance  of  AR-methods  with  noisy  data  also  causes 
inferior  frequency  estimates  at  low  SNR.  In  order  to  alleviate  this  problem,  a  large  class  of  methods  based  on 
principal-component  (PC)  analysis,  have  been  developed  for  reducing  the  effect  of  noise  in  data  [3,  7].  But  the 
PC-based  methods,  though  highly  effective  for  tone- frequency  estimation,  can  not  be  used  for  cleansing  noisy 
AR-data.  This  is  because  the  data  and  correlation  matrices  are  theoretically  full-rank  in  this  case  even  when 
there  is  no  observation  noise  at  all.  A  new  class  of  algorithms,  referred  to  as  KiSS  or  IQML,  have  been  developed 
recently  for  Maximum-Likelihood  frequency  estimation  [5,  6].  The  KiSS  algorithm  essentially  prefilters  the  noisy 
data  by  iteratively  minimizing  the  projection  of  the  observations  onto  the  noise  subspace  formed  with  linear 
predictor  type  polynomial  coefficients.  It  is  shown  in  this  work  that  this  matrix-prefiltering  approach  also  has  the 
desired  noise-reduction  effect  on  pure-AR-in-noise  data.  Extensive  simulation  studies  indicate  that  the  proposed 
prefiltering  produces  more  accurate  AR-spectra  than  the  conventional  AR-modeling  approaches. 
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Formulation  of  Data-Adaptive  Prefiltering 

The  proposed  approach  may  be  best  explained  by  outlining  the  initialization  step  and  the  noise-subspace 
projection  utilized  by  the  KiSS-IQML  algorithm  [5,  6].  In  that  algorithm,  the  frequency  estimation  problem  is 
essentially  transformed  into  an  AR-type  polynomial  estimation  problem.  Specifically,  let, 

B(z)  A  b0  +  b1z~ 1  -f  ...  +  bpz~p  (1) 

be  a  pth  degree  ^-polynomial  with  roots  at  ejw> ,  eJW2  . . .  ,  respectively.  The  coefficient  vector, 

b  A  [60  h  ■■■  bp]T  (2) 

is  estimated  by  minimization  of  the  following  error  criterion  : 


min  b^X^BB^^Xb  where,  (3a) 


f bp  ...  bo 

°\ 

and 

\  0  bp 

...  b0 J 

*(P)  1 

x(p  +  1)  | 

x(p  ~  1) 
x{p) 

*(0)  \ 
z(l) 

U(AT-l)  | 

x(N-  2) 

. . .  x(N  —  p  —  1)  / 

A  (g  |  G). 


(36) 


(3c) 


(3d) 


The  weighted-quadratic  structure  in  (3a)  is  utilized  for  minimizing  the  criterion  iteratively.  At  the  ( i  +  l)-th 
iteration,  the  weight  matrix  (BBH )  is  formed  with  the  estimate  of  b  found  at  the  i-th  iteration  and  the  following 
criterion  is  minimized  to  obtain  the  updated  estimate  : 


min  bH[Xff(BWBw(i))-1X]b. 


(4) 


The  iterative  algorithm  in  (4)  is  initialized  with,  b  =  [10  ...  0]T.  Hence,  the  initial  estimator  has  the  following 
form  : 


min  b^X^Xb  (5a) 

=  min  ||Xb||2.  (56) 

Interestingly,  this  criterion  is  exactly  identical  to  the  'Covariance  Method’  of  linear  prediction  used  in  AR-modeling 
[4].  If  the  data  contains  no  observation  noise,  the  minimization  in  (5)  would  indeed  produce  exact  frequency 
estimates.  Furthermore,  the  performance  of  covariance  method  for  modeling  pure  AR-processes  without  any 
observation  noise  is  also  known  to  be  quite  good  [4].  But  the  performance  deteriorates  drastically  with  noisy 
observation  data.  In  fact,  simulations  indicate  that  even  at  reasonably  high  SNR  of  30-35  dB,  Covariance  (or 
Autocorrelation)  method  may  not  be  able  to  distinguish  closely  spaced  peaks  or  frequencies  in  the  underlying 
process. 
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In  order  to  improve  on  the  initial  estimate  obtained  using  (5),  the  criterion  in  (4)  is  iteratively  minimized  in 
KiSS/IQML.  But,  the  original  criterion  in  (3a)  also  has  the  following  equivalent  forms  : 


bffX/f(BB/f)-1BBff(BBff)-1Xb 

(6a) 

x"BJ?(BBH)-1BBff(BBJIf)-1Bx 

(66) 

xsPbwPbwX 

(6c) 

I|Pb*x||2 

(6  d) 

||Bff(BBff)_1Bx||2 

(6e) 

||WBx||2 

(6/) 

||WXb||2, 

(6ff) 

ection  matrix’  of  B^, 

A  Bff(BBff)_1 

(7a) 

*(1)  •••  *(W-1)]T], 

(76) 

is  a  weighting  matrix  and 


is  the  observation  vector. 

Equation  (6d)  shows  that,  in  order  to  reduce  the  effect  of  noise  in  this  AR-type  parameter  estimates,  the 
projection  of  the  data  (x)  onto  the  column-space  of  the  matrix  needs  to  be  minimized.  The  criterion 
in  equation  (6g)  is  similar  to  the  criterion  in  (5b)  for  Covariance  method,  except  that  in  (6g)  the  projection 
operation  essentially  prefilters  the  data  matrix  X  by  the  weight  matrix  W  which  is  formed  by  the  coefficients 
estimated  at  the  previous  iteration  step.  The  most  obvious  conclusion  from  this  discussion  is  that  the  noise- 
suppression  capability  of  KiSS-IQML  is  essentially  due  to  this  prefiltering  of  the  data-matrix  (X)  which  appears 
in  conventional  Covariance  method  for  AR  modeling. 

As  mentioned  before,  multiple  sinusoids  can  be  modeled  as  a  limiting  case  of  narrowband  AR-process  [1].  The 
analogies  noted  above  appears  to  lead  to  the  possible  hypothesis  that  similar  prefiltering  operation  may  be  equally 
effective  in  reducing  noise  effects  on  the  AR  parameter  estimates  also,  especially  for  narrowband  AR-processes. 
The  algorithm  outlined  next  essentially  minimizes  the  projection  of  the  data  onto  the  column-space  of  in 
order  to  obtain  improved  estimates  of  the  AR-parameter  vector  b. 

Steps  for  the  Proposed  Prefiltering  Algorithm 

1  :  Obtain  the  initial  estimate  of  the  AR-parameters  in  b  using  any  of  the  conventional  AR-modeling  methods. 

2  :  Form  the  BW  matrix  defined  in  (3b)  using  the  estimate  of  b  found  in  the  previous  iteration. 

3  :  Minimize  the  criterion  (4b)  to  obtain  an  updated  estimate  of  b,  which  has  the  following  form  : 

1 


b(<+1>  = 


(8) 


where,  B^  denotes  the  matrix  obtained  is  Step-2  whereas,  g  and  G  are  defined  in  (3d). 

4  :  Go  to  Step-2  unless  ||b(,+1)  —  bW||2  <  6,  where  6  is  a  small  number. 

An  important  difference  between  KiSS-IQML  algorithm  and  the  proposed  method  is  that  in  case  of  KiSS- 
IQML,  conjugate-symmetry  constraints  need  to  be  imposed  on  the  coefficients  of  the  i?(,2:)-polynomial  in  an 
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attempt  to  constrain  the  roots  to  lie  on  the  unit  circle.  This  makes  the  optimization  problem  even  more  nonlinear. 
But  in  the  present  case  no  such  constraints  are  necessary  and  hence,  the  optimization  in  (4)  is  more  straight¬ 
forward.  Extensive  simulation  studies  have  shown  that  this  algorithm  does  produce  better  AR  spectrum  match 
at  lower  SNR  than  any  of  the  standard  AR-modeling  techniques. 

Simulation  Results 

The  test  data-set  given  in  Chapter-7  in  [3]  was  generated  for  the  simulation.  Fig.  1.  illustrates  the  per¬ 
formance  of  the  Covariance  method  for  50  independent  realizations  of  the  observation  data  at  30dB  SNR.  The 
solid  line  in  Fig.  2  shows  the  average  of  estimated  spectra  of  the  50  realizations  and  the  dashed  line  shows 
the  true  spectrum.  Figures  3-4  and  5-6  show  the  corresponding  results  with  Modified  Covariance  Method  and 
Autocorrelation  Method,  respectively,  with  identical  data  sets.  The  results  clearly  demonstrate  that  even  at  this 
moderately  high  SNR,  none  of  these  commonly  used  methods  were  able  to  distinguish  the  two  spectral  peaks 
for  most  of  the  noise  realizations.  Fig.  7  shows  the  results  of  the  proposed  prefiltering  algorithm  for  those  50 
identical  realizations  at  the  same  SNR.  The  iterations  converged  in  6-8  iterations  in  all  cases.  Fig.  8.  shows  the 
average  of  the  50  realizations  with  the  true  spectrum.  This  improvement  was  found  to  be  consistent  even  at  lower 
SNR  values.  Similar  improvements  have  also  been  observed  when  the  Auto-correlation  method  and  the  Modified 
Covariance  method  were  used  to  generate  the  initial  AR  parameter  estimates.  The  plots  clearly  demonstrate 
that  the  proposed  method  was  able  to  match  the  AR-spectra  more  closely.  With  simulated  data,  the  average 
prediction  error  power  for  the  proposed  estimator  was  also  found  to  be  much  smaller  than  the  standard  methods. 
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Section  -  2.4  :  IMPROVED  ARMA-PARAMETER  ESTIMATION  FROM  NOISY  OBSERVATION  DATA 


Summary 

Existing  methods  for  ARMA  modeling  assume  that  the  available  process  is  produced  by  an  ARMA  system 
driven  by  a  white  input  process,  i.e.,  the  observed  process  is  considered  to  be  pure  ARMA.  In  practice,  the 
available  data  usually  have  observation  noise  added  to  it  but  the  ARMA  methods  do  not  address  this  problem. 
Simulations  show  that  performance  of  the  existing  ARMA  methods  deteriorate  when  the  observation  process  is 
noisy.  In  this  Section  a  new  ARMA  algorithm  is  given  which  utilizes  a  recently  developed  deterministic  rational 
system  identification  method  (OM-IO)  [8]  that  minimizes  the  modeling  or  output  error  norm.  The  algorithm  first 
estimates  the  input  process  and  then  invokes  OM-IO  using  the  input-output  data.  Simulations  indicate  that  the 
proposed  method  is  quite  effective  even  at  low  SNR  observation  data. 

Introduction 

With  both  poles  and  zeroes,  ARMA  models  are  best  capable  of  effectively  representing  general  spectra  with 
possibly  sharp  peaks  as  well  as  deep  valleys.  Modeling  of  ARMA  processes  involves  solution  of  a  set  of  highly 
nonlinear  equations.  Existing  methods  divide  the  problem  into  several  ‘equation  error*  minimization  problems 
to  estimate  the  AR  and  MA  parameters  in  several  stages.  The  estimation  problem  is  further  complicated  if  the 
available  data  is  also  corrupted  with  observation  noise.  In  fact,  simulation  studies  indicate  that  the  performance  of 
the  existing  ARMA  modeling  methods  deteriorate  significantly  with  noisy  data.  This  drawback  may  be  attributed 
to  the  sensitivity  of  equation-error  minimization  based  methods  to  the  presence  of  noise.  In  this  Section,  we 
propose  to  address  this  problem  by  incorporating  a  recently  developed  optimal  algorithm  for  identification  of 
deterministic  ARMA  systems  [8]  into  the  stochastic  ARMA  modeling  problem.  Recent  results  indicate  that 
estimators  based  on  minimizing  model  ‘fitting-error *  have  superior  performance  when  compared  to  those  which 
rely  on  equation-error  minimization  [8,  13].  In  view  of  this,  unlike  existing  ARMA  methods,  the  algorithm 
presented  in  this  work  minimizes  output  or  modeling  errors.  The  results  obtained  so  far  indicate  that  the  proposed 
approach  is  much  more  effective  than  existing  methods  for  ARMA  parameter  estimation  when  the  available  data 
is  not  purely  ARMA  but  has  some  observation  noise  added  to  it. 

The  ARMA  Model 

An  ARMA (p,q)  process  can  be  represented  in  a  linear  difference  equation  form  as, 

p  q 

x(n)  =  —  —  k)  +  ^2  aku(n  ~  &)  (1) 

k= 1  k= 0 


where,  the  corresponding  z-domain  transfer  function  has  the  following  form  : 


>.  ao  +  a\z  1 -f  •  •  • -| -  aqz  q 

1  +  6i2_1  +  62Z-2  H - \-bpZ~P  = 

A{z) 

(2) 

B(z) 

Let, 

a  A  [ao  a\  •  •  •  aq]T  and 

(3a) 

b  A  [1  &i  •  •  •  bp]T 

(36) 

denote  the  unknown  MA  and  AR  parameters,  respectively.  In  vector  form, 

x  A  [a?(0)  z(l)  •  •  •  x(N  —  1)]T  and 

(4a) 

u  A  [u( 0)  u(l)  u(N  —  1)], 

(46) 
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denote  the  output  data  and  the  driving  noise  sequences,  respectively. 

Previous  Methods 

Estimation  of  the  Maximum- Likelihood  (ML)  parameters  of  an  ARM  A  process  is  a  highly  nonlinear  problem. 
Akaike’s  MLE  method  requires  nonlinear  optimization  which  are  prone  to  poor  convergence  if  the  initial  estimates 
are  chosen  properly  [4].  To  overcome  the  complexities  of  MLE,  many  computationally  attractive  techniques  have 
been  developed  also.  Among  these,  the  Modified  Yule-Walker  equations  (MYWE)  method  estimates  the  AR 
parameters  from  the  tail-end  of  Yule- Walker  (Y-W)  equations,  Le.,  from,  r^fc),  k  =  q  +  1, . . . ,  q  +  p.  The  output 
process  is  then  filtered  by  the  estimated  A(z)  filter,  which  results  in  an  MA  process  from  which  the  MA  parameters 
can  be  determined  by  any  standard  procedure  for  MA  parameter  estimation  [9].  An  extension  of  this  approach, 
known  as  the  Least-Squares  MYWE  (LSMYWE)  [10],  uses  more  of  the  tail-end  of  Y-W  equations  and  yields 
better  results  than  MYWE. 

In  stochastic  ARM  A  modeling,  the  driving  white  noise  sequence  is  completely  unknown.  Clearly,  if  it  were 
somehow  possible  to  have  some  estimate  of  the  driving  noise  u(n),  then  any  input-output  system  identification 
technique  could  be  used  to  estimate  the  ARMA  parameters.  Two  well-known  ARMA  methods  are  indeed  based 
on  this  principle,  namely,  Two-Stage  Least-Squares  [5]  and  Three-Stage  Least-Squares  [7,  12].  The  primary  steps 
in  these  methods  are  to  model  the  output  data  first  as  a  large  order  AR  process,  then  a  prediction  error  sequence 
is  obtained  by  passing  the  data  through  the  inverse  filter  which  is  MA.  This  whitened  prediction  error  sequence 
is  used  as  the  estimate  of  the  input  white  noise  sequence  u(n).  With  this  estimated  input  and  the  observed 
output,  the  ARMA  parameters  are  then  found  by  minimizing  equation  errors  in  two  [5]  or  three  stages  [7],  The 
three-stage  approach  has  been  shown  to  have  lower  variance  than  the  two-stage  case.  But,  as  will  be  shown  with 
simulations  below,  even  the  three-stage  algorithm  can  not  perform  well  when  the  observation  data  is  noisy,  which 
is  quite  possible  in  practical  situations. 

It  may  be  mentioned  here  that  in  [11]  a  data-adaptive  prefiltering  method  has  also  been  proposed  for  improved 
modeling  of  AR-parameters  from  noisy  observation  data.  As  noted  in  [11],  there  have  been  some  previous  work  on 
AR-modeling  from  noisy  data,  but  the  author’s  are  not  aware  of  any  such  work  for  modeling  ARMA  parameters 
from  noisy  data,  which  is  the  problem  considered  in  this  work. 

The  Proposed  Idea 

Instead  of  minimizing  the  equation  error  criterion  as  in  [5,  7],  the  proposed  algorithm  minimizes  the  modeling 
error  or  output  error  criteria.  This  is  also  a  nonlinear  problem,  but  a  recently  developed  input-output  identification 
method  optimally  decouples  the  numerator  and  denominator  problems  [8]  into  two  separate  problems  of  smaller 
dimensions.  The  decoupled  estimators  retain  the  global  optimum  of  the  original  criterion.  It  has  been  further 
shown  in  [8]  that  in  the  decoupled  form,  estimation  of  the  numerator  a  is  a  purely  linear  problem  whereas  the 
estimation  of  the  denominator  is  a  nonlinear  problem  of  reduced  dimensionality.  But  the  nonlinear  criterion  for 
the  denominator  possesses  a  convenient  weighted-quadratic  structure  which  can  be  easily  exploited  to  estimate  the 
denominator  iteratively.  Preliminary  simulation  studies  show  that  the  proposed  method  outperforms  the  existing 
ARMA  modeling  approaches  when  the  observed  data  is  corrupted  with  noise.  Brief  explanation  of  the  underlying 
theory  along  with  the  algorithm  steps  are  in  order.  Some  simulation  results  included  at  the  end  demonstrate  the 
superior  performance  of  the  proposed  method. 

FORMULATION  OF  THE  ESTIMATION  PROCEDURE 

Let, 

y(n)  =  a;(n)  +  v(n),  (5) 

be  the  observed  noisy  ARMA  process,  where  v(n)  denotes  the  observation  noise  process.  Let, 

y  a  [y(0)  y(l)  •  •  •  y(N  -  1)]T  (6) 
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denote  the  noisy  output  vector.  Using  covariance  method,  this  observation  data  is  first  modeled  as  a  large  order 
(=L)  AR  model  to  obtain  an  AR-polynomial  BL(z)  such  that  L  »  p,  the  true  AR-order  of  the  underlying 
ARM  A  process.  The  observation  sequence  y(n)  is  then  filtered  through  an  MA-filter  in  the  form  of  BL(z)  to 
obtain  a  whitened  prediction  error  sequence  u(n).  Then  using  u(n)  as  the  estimate  of  the  input  sequence,  the 
ARMA  modeling  problem  can  be  restated  as  the  following  output-error  minimization  problem  : 

mjn  ^2  b(0  -  ^4{«(0}]2-  (7) 

It  has  been  shown  in  [8]  that  the  above  nonlinear  problem  can  be  decoupled  into  a  purely  linear  problem  to 
estimate  a  and  a  nonlinear  problem  for  b.  Such  decoupling  techniques  have  also  been  found  to  be  very  effective 
in  Maximum  Likelihood  estimation  of  the  parameters  of  multiple  exponential  models  [2,  3].  It  may  be  noted  here 
that  the  estimators  in  [6,  7]  utilize  the  estimated  prediction  error  sequence  u(n).  But  those  estimators  do  not 
minimize  the  true  model-fitting  defined  in  [7],  but  are  based  on  minimizing  equation  error  norms.  The  following 
definitions  are  necessary  to  formulate  the  decoupling  of  the  numerator  and  denominator  optimization  problems. 

Let  Ht(z)  be  an  inverse  filter  corresponding  to  B{z))  i.e 

B(z)Hb(z)  =  1.  (8) 


By  writing  this  convolution  in  matrix- form,  it  can  be  shown  that  [8], 
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which  leads  to  the  conclusion  that  BT  is  orthogonal  to  the  matrix  H&.  Utilizing  this  orthogonality  relationship, 
the  optimal  criterion  for  estimating  the  denominator  can  be  shown  to  be  [8]  : 

min  y^UfB(Bi?U/UfB)-1Bi/U/y 

b 

=  min  bffZJ?(B/fU/UfB)"1Zb 


where,  U  is  a  lower-triangular  convolution  matrix  formed  with  the  estimated  input  sequence  u(n), 
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U  A 


6(1) 


u(0) 
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The  matrix  U/  is  the  inverse  of  U  and  is  also  lower  triangular. 
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z  A  U/y  and 
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where,  the  matrix  Z  has  the  following  Toeplitz  structure, 
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where,  g  is  a  column  vector  formed  with  the  leading  column  of  Z.  The  denominator  vector  b  is  estimated 
by  minimizing  the  criterion  in  (10).  The  minimization  is  performed  iteratively  by  forming  the  weight-matrix 
(B^U/U^B)  with  the  estimate  of  b  obtained  at  the  previous  iteration  step.  At  convergence  of  the  iterations, 
the  estimated  b  is  used  to  form  the  matrix  H&  using  its  inverse  sequence  as  in  (8).  Then  the  least-squares  solution 
of  the  numerator  a  is  found  as, 

a  A  (UH6)#y  (12) 

where,  #  denotes  matrix  pseudo-inverse.  The  iterative  process  is  initialized  by  estimates  obtained  by  minimizing 
equation  errors  as  in  [6,  7,  10].  Hence,  the  further  iterations  of  the  proposed  method  can  only  improve  upon  the 
equation-error  based  estimates  because  it  minimizes  the  true  modeling  error  criterion  defined  in  (7). 

The  Overall  algorithm  in  Brief 

The  complete  algorithm  for  the  ARMA  parameter  identification  can  be  summarized  as  the  following  four 
primary  steps  : 

1.  Model  the  observed  sequence  y  by  a  large  order  AR  model. 

2.  Determine  the  prediction  error  white  noise  sequence  u,  which  is  treated  as  the  input  sequence  for  the  system. 

3.  Knowing  u  and  y,  start  the  iterative  procedure  to  minimize  the  error  criterion  in  (10).  At  each  iteration, 
b  is  estimated  either  as  the  eigenvector  corresponding  to  the  minimum  eigenvalue  of  ZH (BHXJ iUf'B)~1Z 
or  by  setting  bo  =  1.  In  the  later  case,  the  estimate  of  b  at  the  (i  +  l)-th  iteration  is  obtained  using  the 
estimates  of  the  previous  iteration  step  as  follows, 

1 

b(i+1)  =  .  ,  (13a) 

,-{Gh  wff<i>W(*)G)-1Gff  Wtf  WwMg . 

where,  the  matrix  W  is  formed  with  the  estimates  of  b  at  the  previous  iteration  step  as, 

W  A  Uf(B/fU/UfB)_1.  (136) 

The  iterations  are  continued  till  convergence  is  reached,  i.e.,  no  significant  change  is  found  in  b  between 
successive  iterations. 

4.  Estimate  a  using  (12). 
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Simulation  Results 

The  simulations  were  performed  with  the  test  data  set  (ARMA4)  used  in  Chapter- 10  of  [1].  The  true  system 
PSD  has  two  prominent  peaks  as  shown  in  Fig.  1.  The  PSD  estimates  at  an  SNR  of  20dB  in  the  observation 
data  y(n)  are  shown  in  Fig.  2.  The  results  using  MYWE,  LSMYWE,  Maine-Firoozan  and  the  proposed  method 
are  shown  in  Figures  2a  through  2d.  Clearly  the  proposed  method  performs  better  than  the  other  three.  The 
corresponding  results  with  15dB  SNR  are  shown  in  Fig.  3a  -  3d.  The  performance  of  the  proposed  method 
is  maintained  even  at  this  level  of  SNR,  though  the  results  with  the  three  existing  methods  have  deteriorated. 
Further  simulations  at  lower  SNR  levels  indicate  that  the  peaky  spectral  shape  is  maintained  at  least  up  to  12dB. 
The  efficacy  of  the  algorithm  is  obvious. 

References 

[1]  S.  M.  Kay,  Modern  Spectral  Estimation :  Theory  and  Applications ,  Prentice-Hall,  Englewood  Cliffs,  NJ,  1988. 

[2]  R.  Kumaresan,  L.  L.  Scharf  and  A.  K.  Shaw,  “An  Algorithm  for  Pole-Zero  Modeling  and  Spectral  Estimation” , 
IEEE  Transactions  on  Acoustics ,  Speech  and  Signal  Processing ,  vol.  ASSP-34,  pp.  637-640,  June  1988. 

[3]  Y.  Bressler  and  A.  Macovski,  “Exact  Maximum  Likelihood  Parameter  Estimation  of  Superimposed  Expo¬ 
nential  Signals  in  Noise”,  IEEE  Transactions  on  Acoustics,  Speech  and  Signal  Processing ,  vol.  ASSP-34,  no. 
10,  pp.  1081-1089,  Oct.  1988. 

[4]  H.  Akaike,  “Maximum  Likelihood  Identification  Gaussian  Autoregressive  Moving  Average  Models,” 
Biometrika ,  vol.  60,  pp.  255-265,  1973. 

[5]  J.  Durbin,  “The  Fitting  of  Time-Series  Models,”  J.  Roy .  Stat.  Soc .,  vol.  22,  pp  233-243,  1960 

[6]  T.  C.  Hsia,  System  Identification ,  D.  C.  Heath,  Lexington,  MA,  1977. 

[7]  D.  Q.  Mayne  and  F.  Firoozan,“An  Efficient,  Multistage,  Linear  Identification  Method  for  ARMA  Processes,” 
Proceeding  of  IEEE  Conference  on  Decision  and  Control ,  New  Orleans,  Vol.  1,  pp.  435-438,  Dec.  1977. 

[8]  A.  K.  Shaw,  “A  New  Algorithm  for  Optimal  Estimation  of  Plant  Parameters  from  Input-Output  Data,”  31st 
IEEE  Conference  on  Decision  and  Control ,  Tucson,  AZ,  pp.  1684-1685,  Dec.,  1992. 

[9]  J.  Durbin,  “Efficient  Estimation  of  Parameters  in  Moving  Average  Models,”  Biometrika  vol.  46,  pp  306-316, 
1959. 

[10]  J.  Cadzow,  “High  Performance  Spectral  Spectral  Estimation  -  A  New  ARMA  Method,”  IEEE  Transactions 
on  Acoustics,  Speech  and  Signal  Processing ,  vol.  ASSP-28,  pp  524-529,  Oct.  1980. 

[11]  A.  K.  Shaw  and  S.  Kundu,  “AR-Spectrum  Estimation  from  Noisy  Observation  Data,”  Submitted  for  Review, 
27th  Asilomar  Conference  on  Signals,  Systems,  and  Computers . 

[12]  I.  S.  Konvalinka  and  M.  R.  Matausek,  “Simultaneous  Estimation  of  Poles  and  Zeros  in  Speech  Analysis  and 
ITIF-Iterative  Inverse  Filtering  Algorithm,”  IEEE  Transactions  on  Acoustics,  Speech  and  Signal  Processing , 
vol.  ASSP-27,  pp  485-492,  Oct.  1979. 

[13]  A.  K.  Shaw,  “Optimal  Estimation  of  the  Parameters  of  All-Pole  Transfer  Functions,”  IEEE  Transactions  on 
Circuits  and  Systems ,  vol.  41,  no.  2,  pp. 


58 


59 


Maenitude(dB) 


Section  -  2.5  :  TIME-DOMAIN  DETECTION  OF  ELECTRONIC  WARFARE  SIGNALS  IN  NOISE 


Summary 

In  the  passive  mode  of  operations  of  EW  applications,  source  signals  may  not  be  present  within  a  given 
observation  window,  or  the  signals  may  fill  only  a  part  of  the  estimation  window.  In  either  case,  any  frequency 
estimation  algorithm  may  produce  erroneous  or  noise  frequencies.  Considering  the  relatively  high  computational 
burden  of  any  frequency  estimation  method,  it  is  desirable  to  invoke  a  frequency  estimation  method  only  when 
a  detection  scheme  indicates  high  probability  of  presence  of  threat.  In  this  Section,  the  theory  of  detection  of 
sinusoids  from  Quantized  and  Noisy  time-domain  observation  samples  have  been  developed.  The  theoretical  work 
on  single/multiple  samples  is  mostly  complete.  Studies  with  Quantized  data  have  also  been  performed  and  the 
results  appear  reasonably  good.  Lab  tests  for  the  Envelope  Detection  and  Square-Law  cases  have  been  conducted 
at  Wright  Labs  with  satisfactory  results. 

I.  Introduction  : 

In  Electronic  Warfare  (EW)  environments,  microwave  receivers  play  a  major  role  in  passive  identification  and 
localization  of  unknown  targets  emitting  high-frequency  electro-magnetic  signals.  EW  signals  cover  a  relatively 
wide  bandwidth,  typically  in  the  range  of  0.2  to  15  GHz,  and  existing  microwave  receivers  utilize  mostly  analog 
signal  processing  tools  and  techniques  [1-3].  In  fact,  there  are  no  EW  receivers  that  process  microwave  radar  signals 
entirely  in  the  digital  domain.  With  the  emergence  of  increasingly  faster  and  inexpensive  digital  computers  and 
high-speed  A/D  converters,  it  is  expected  that  digital  processing  of  microwave  radar  signals  is  expected  to  be 
practically  feasible. 

The  primary  task  of  a  microwave  receiver  is  to  gather  data  for  sorting  of  signals  and  identification  of  the  type 
of  the  radar  emitting  the  received  signal.  Based  on  these  information  jamming,  weapon  delivery  or  other  options 
are  considered.  In  order  to  perform  these  tasks,  the  receiver  must  analyze  the  received  radar  pulses  and  measure  or 
estimate  the  following  six  parameters  :  Angle-of- Arrival  (AOA),  Radio  Frequency  (RF),  Time  of  Arrival  (TOA), 
Pulse  Amplitude  (PA),  Pulse  Width  (PW)  and  Polarization  (P).  But  in  order  to  reduce  computational  burden,  the 
estimation  of  these  parameters  should  be  undertaken  only  when  it  is  determined  that  there  is  a  high  probability 
of  the  presence  of  a  threat  signal. 

In  this  part  of  the  project,  the  detection  problem  has  been  considered  in  the  time-domain  for  single  and 
multiple  samples.  Detection  thresholds  and  Probability  of  Detection  based  on  Ney man- Pearson  Criterion  have 
been  derived.  Derivations  are  given  for  calculating  the  Thresholds  and  Probability  of  Detection  for  both  the 
‘Square-Law’  and  ‘Envelope’  detectors. 

II.  Time-Domain  Detection  : 

Almost  all  existing  AOA/RF  estimation  algorithms  assume  that  the  signal  is  already  present  in  the  observed 
data.  But  in  the  passive  mode  of  operations  of  EW  applications,  source  signals  may  not  be  present  at  all  within 
the  observation  window,  or  the  signals  may  fill  only  a  part  of  the  estimation  window.  In  either  case,  any  frequency 
estimation  algorithm  would  essentially  produce  erroneous  or  noise  frequencies  because  the  observed  signal  would 
not  satisfy  the  model  assumed  by  the  estimation  algorithm.  Considering  the  relatively  high  computational  burden, 
any  estimation  method  should  be  invoked  only  when  a  detection  scheme  indicates  high  probability  of  presence  of 
threat. 

Since  EW  receivers  do  not  have  any  prior  knowledge  about  the  frequency /amplitude/phase  of  the  received 
signals,  conventional  matched  filters  can  not  be  used  in  this  case.  An  obvious  solution  would  be  to  perform  the 
detection  in  the  frequency-domain,  i.e.,  the  presence  of  targets  can  be  determined  by  thresholding  of  FFT-peaks. 
The  frequency-domain  approaches  are  robust  but  have  certain  disadvantages  in  that  a  decision  can  be  made  only 
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after  a  block  of  data  has  been  collected.  Furthermore,  a  lot  of  computational  power  may  be  wasted  if  FFT  is  taken 
continuously,  even  when  no  target  is  present.  Instead,  we  plan  to  incorporate  a  time-domain  detection  scheme 
that  can  detect  targets  in  real-time  using  a  single  observation  or  a  small  number  of  samples.  Once  a  preliminary 
decision  is  taken,  FFT  or  more  sophisticated  frequency/AOA  estimation  algorithm  can  be  invoked,  if  desired. 

II.  1  :  Signal  and  Noise  Model 

Microwave  radars  signals  can  be  modeled  as, 

x(n )  =  Acos(ojck  +  0)  +  n(k)  (la) 

=  Acos(u)ck)cos6  -  Asin(u;cfc)  sin  0  +  ni(k)  cos(uck)  +  riQ(k)  sin(uck)  (16) 

where,  n(k)  denotes  narrowband  noise  samples.  To  perform  the  time-domain  detection,  the  received  real  data 
is  first  converted  into  a  complex  analytic  signal.  This  is  achieved  by  passing  the  real  signal  through  a  Hilbert 
Transformer  to  form  the  in-phase  (I)  and  quadrature  (Q)  components  of  the  complex  analytic  signal.  When  no 
signal  is  present,  the  I  and  Q  components  may  be  represented  as, 

Xi(k)  =  m(k)  (2a) 

XQ(k)  =  nQ(k).  (26) 

On  the  other  hand,  in  the  presence  of  signal,  the  corresponding  components  are  given  as  : 

Xf(k)  =  AcosO  +  ni(k)  (3a) 

Xq(&)  =  Asin0  +  riQ(k).  (36) 

Since  the  amplitude,  frequency  and  the  phase  of  the  received  signal  are  unknown,  the  detection  criterion  has 
to  rely  on  thresholding  of  the  amplitude  (PA)  of  the  analytic  signal.  The  frequency  and  phase  can  be  ignored 
for  detecting  only  the  presence  of  a  target  signal.  The  amplitude  threshold  can  not  be  based  on  minimizing  the 
total  probability  of  error  because  the  exact  amplitude  of  the  signal  is  unknown  at  the  receiver.  Furthermore,  the 

probability  of  False  Alarm  (Pfa)  must  also  be  kept  very  low  (10“6  or  smaller).  Hence,  the  best  detection  scheme 

would  be  to  calculate  the  threshold  by  setting  the  Pfa  to  a  constant.  The  thresholds  for  Square-Law  detector 
have  been  derived  next  for  single  and  multiple  samples  within  a  pulse. 


II. 2  :  Square  Law  Detector 

The  noise  is  assumed  to  be  narrowband  and  Gaussianly  distributed  with  zero-mean  and  variance  —  <x2. 
Hence,  for  the  no-signal  case  of  (2)  the  I/Q  noise  samples  are  distributed  as  : 

Xj(k)  =  N(  0,cr2)  (4a) 

XQ  (*)  =  N(0,<r2).  (46) 

In  the  following  derivation,  the  time- variable  k  will  be  suppressed  until  the  multiple  samples  case  is  considered. 


II.2.a  :  Single  Sample  Case 


Assuming  independent  noise  samples,  when  no  signal  is  present,  the  joint  probability  density  function  (PDF) 
of  the  I/Q  channel  outputs  are  given  by  : 


A*/.**)  =  + 

(5) 

Xj  =  Rc os  a 

(6a) 

Xq  =  Esina. 

(66) 
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(7) 


Using  the  Jacobian  of  this  transformation,  the  joint  PDF  for  this  polar  form  can  be  shown  to  be  : 

f(r,a)  = 


r  r2 

e  2 


2  ttct2 


From  this  the  marginal  for  the  Envelope  (R)  is  given  by, 

[2*  V  _r* 

/fl(r)  =  J J  f(r,a)dot  -  e  ^ u(r ), 


which  is  known  as  the  Rayleigh  PDF. 

II.2.a.l  :  The  PDF  and  Characteristic  Function  with  No  Signal 


(8) 


A  square-law  detector  forms  the  following  quantity, 


Z  A  X]  +  Xq  =  R2 

(9) 

which  needs 

M  =  2R  = 

to  be  compared  to  a  threshold  to  decide  the  presence/absence  of  a  radar  target. 

2 \/~Z ,  the  PDF  of  the  Square-Law  output  when  no  signal  (denoted  as,  s)  is  given  as  : 

Since, 

f  ,  ,_x  fnWz)  1  , 

M*l>)  =  2Vj  =  =•«('). 

(10) 

which  is  the  Exponential  PDF.  The  Characteristic  Function  (CF)  is  defined  as  the  Fourier  Transform  of  the 
Density  function  : 


Cfr(w)  A  T[fz{z\s)] 

=  —  f 

2<t2  J_ 


e~5^  e~jwzdz 


1 


1  +  j2u><T2 


(11) 


II.2.a.2  :  The  PDF  and  Characteristic  Function  in  Presence  of  Signal 

When  target  is  present,  2.e.,  in  case  of  (3),  the  I/Q  samples  are  distributed  as  : 

Xj(fc)  =  N(A  cos  0,  <x2) 

XQ(k)  =  N(A  cos  6,  a2). 

In  this  case,  the  joint  probability  density  function  (PDF)  is  given  by  : 

f(Xl,XQ)  =  — e-^*'  ~  ^cosO)2  +  (xQ  -  ylsin#)2] 

27T<T2 

Once  again,  using  the  Jacobian  of  the  transformation,  the  polar-form  joint  PDF  can  be  shown  to  be  : 

f(r, a|s)  =  -i-e-3^[r3  +  A*  ~  2>lrcos(“  "  ^(r). 

Integrating  over  a,  the  marginal  PDF  of  the  Envelope  is  given  by, 


/fl(Hs) 


2tt 

JO 


/(r,  a\s)da 


27 rcr2 


e +  r2)  C*  ~  °Ua, 

Jo 


=  V^2  +  r3)/of^ 


(12a) 

(126) 

(13) 

(14) 

(15a) 

(156) 

(15c) 
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where,  /o(-)  denotes  Bessel  Function  of  the  zero-th  kind.  The  PDF  in  (15c)  is  known  as  the  Rician. 

Similar  to  the  no-signal  case  in  (9)-(10),  the  PDF  of  the  Square-Law  output  Z  with  signal-plus-noise  is  given 

as  : 

/z(2|s)  =  “VT  =  )  <16> 

In  this  case,  the  Characteristic  Function  can  be  found  as  follows  : 


C£(w)  A  T[fz{z |,)] 


e  2^  /0 


e~JW  dz 


The  following  Fourier  Transform  pair  can  be  found  in  [CF,  page-79,  pair  655,1]  : 


T  e'^/n 


-eA(w  +  . 


Using  (18)  and  with  appropriate  substitutions,  C^(a;)  is  given  by  : 

c*(w)  =  r+j-w6"1  (19) 

II.  2  .a.3  :  The  Ney  man- Pearson  Criterion  with  a  Single  Sample  : 

For  this  one-dimensional  case,  the  decision  that  the  signal  is  present  is  taken  if  the  likelihood-ratio  [17]  : 

*  =  Sum >  HPfa)  (20) 

where  k  is  a  constant  that  depends  on  the  probability  of  False- Alarm  Pfa •  From  this  relationship  it  may  appear 
that  in  order  to  find  the  decision  threshold,  one  would  need  to  know  or  estimate  the  signal.  But  one  of  the  most 
attractive  consequence  of  Ney man- Pearson  criterion  is  that  for  a  given  predetermined  Pfa,  the  threshold  can  be 
set  by  integrating  f(z\s)  over  the  region  where  the  signal  is  present  [11,  17]. 

II.2.a.4  :  Probability  of  False  Alarm 

If  the  threshold  is  denoted  as  7,  the  false-alarm  probability  can  be  calculated  as, 


Pfa  = 


f 

=  /  fz(z\s)dz 

J  7 

1  f°° 

=  — -  J  e~^dz  from  (10), 


II.2.a.5  :  Detection  Threshold 

Taking  natural  logarithm  of  both  sides  of  (21),  the  detection  threshold  is  given 

7  =  —  2cr2  In  PpA. 


II.2.a.6  :  Probability  of  Detection 
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If  the  square-law  output  z  of  is  greater  than  7  from  (22),  then  the  decision  is  taken  that  source  target  is 
present.  Hence  the  probability  of  detection  can  be  calculated  from  : 


r  OO 

Pb  =  /  fz(z\s)dz 

j  7 

[  fz(z\s)dz 
Jo 


=  1 


.  1  -<  r  —hr  (AV* 


dz . 


By  letting,  v  “  &  and  with  appropriate  substitutions, 

1  a 2  f  y/ 2J2  2  / 

Pb  =  1  -  j  ve~v  I0[l 


A2 

2 tr2  ' 


v  Id?;. 


(23) 


(24) 


But  this  integral  possesses  the  form  of  an  Incomplete  Toronto  Function  [13,  pp-348]  which  is  defined  as  follows  : 


fB 

Tjg(m,n,r)  A  2rn“m+1e"r2  /  tm~n In(2rt)dt 
Hence  Pb  can  be  written  in  a  more  compact  form  as  : 


pb  =  1  -  t 


(25) 


(26) 


11.2. b  :  Multiple  Samples  Case 

The  detector  performance  can  be  expected  to  improve  if  the  decisions  can  be  based  on  multiple  observations 
within  a  pulse.  The  question  would  then  be  how  to  combine  the  multiple  samples  in  order  to  come  up  with  an 
inference.  For  the  Envelope  Detection  case,  Tsui  and  Sharpin  have  recently  derived  an  M-out-of-jV  scheme  where 
the  presence  of  target  is  decided  if  at  least  M  samples  out  of  a  total  of  N  exceed  the  detection  threshold  [12]. 
In  this  work  we  take  a  different  approach  where  decisions  are  taken  based  on  the  sum  of  N  squared  samples. 
This  approach  is  more  akin  to  traditional  CW  detection  schemes  where  integration  over  N  pulses  is  performed 
for  making  a  decision  [13]. 

Let  Y  be  the  random  variable  formed  with  the  sum  of  N  independent  squared  samples,  2.e., 

N 

y  A  ;>>(*),  (27) 

k  —  1 

where,  the  PDF  and  CF  of  Z{k)  were  derived  in  II. 2. a. 

11.2. b.l  :  The  PDF  and  Characteristic  Function  of  Y  with  No  Signal 

When  no  signal  is  present,  the  PDF  of  Y  which  is  formed  as  the  sum  of  N  independent  samples,  is  given  by 
the  following  convolution  : 

fY(y\s)  Afz(zi\t)  *  fz(z2\s )  *  ...  *  fz{zN\s )  (28) 

where,  each  of  the  Z(ky s  has  identical  distribution.  Direct  convolution  of  N  PDFs  appears  complicated,  but  it 
is  well-known  that  convolution  in  PDF-domain  implies  multiplication  in  the  CF-domain.  Consequently,  the  CF 
of  Y  is  given  by, 

N 

cm  =  n  cz(“)  =  PzMf 

k=l 

=  (1  +  jW)*  using  (11)  (29) 
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Using  the  inverse  Fourier  Transform  pair-431  [Campbell  and  Foster,  pp-44],  the  PDF  of  Y  is, 

vN~le~ ^ 

=  (2<t2)v (.V  -  l)!t'(^)  (30) 

II.2.b.2  :  The  PDF  and  Characteristic  Function  of  Y  in  Presence  of  Signal 

Using  arguments  similar  to  those  in  the  previous  subsection,  when  signal  is  present,  the  CF  of  Y  is  given  by, 

Cy(w)  =  JJ  C*z(u>)  =  [Cz{u)]N 


(1  +  j2uj<r2)1 


re  1  + 


Once  again,  using  the  inverse  Fourier  Transform  pair-650.0  [Campbell  and  Foster,  pp-77],  the  PDF  of  Y  is, 


fY(y\s)  =  — 2  e~^NA2  +  y)IN- 1 
JYyyi  }  2cr2  N A2 


11.2. b.3  :  The  Ney man- Pearson  Criterion  with  Multiple  Samples  : 

For  this  N-dimensional  case,  the  decision  that  the  signal  is  present  is  taken  if  the  likelihood-ratio  [17]  : 

ty  =  /“(||)  > *( Pfa) •  (33) 

For  a  given  predetermined  Pfa,  the  threshold  can  be  set  by  integrating  f(y\s)  over  the  region  where  the  signal 
is  present  [11,  17]. 

11.2. b.4  :  Probability  of  False  Alarm 

For  7  denoting  the  threshold,  the  false-alarm  probability  is, 


poo 

=  /  fv{y\s)dy 

J  7 

1  r°°  e~^yN~1 

~  2a*  J,  (N  —  1)! 

in  e~& 

~  "  2a2  J0  (N  - 


dy  from  (30), 


,  1  p  e~^yN-1  J 

2*i  J0  (N-  1)!  dy 


where,  /(•)  denotes  Incomplete  Gamma  Function  which  is  defined  as, 


I(u,t)  A  f 
—  Jo 


uVT+i  e-Vvt 


II.2.b.5  :  Detection  Threshold 

For  a  given  Pfa ,  the  threshold  7  can  be  determined  numerically  with  a  computer  or  using  available 
plots/tables  [13]. 


67 


II.2.b.6  :  Probability  of  Detection 


If  the  sum-of-squares  y  is  greater  than  the  threshold  7  determined  from  (35),  then  the  decision  is  taken  that 
source  target  is  present.  Hence  the  probability  of  detection  can  be  calculated  from  : 


pOO 

pd=  fr(y\s)dy 

=  1  “  /  fv(y\s)dy 

Jo 


=  1 


-  NAf 

e  2<r2 


0£\ 


1  ,,  a=i 


2  a2  \NA 2 


■fs 


e  2<r 


^In- 


(Ay/Ny 

V  v2 


dy 


By  letting,  v 2  =  ^2  and  with  appropriate  substitutions, 


PN 

— 


1  - 


e  2<r2 
^2- 


N_,  _v,t  (  fNA?  \ 

\NAt)  l  V  6  M2V^J 


dv. 


(36) 


(37) 


This  integral  also  possesses  the  form  of  an  Incomplete  Toronto  Function  defined  in  (25).  Hence  P$  can  be  written 
in  a  more  compact  form  as  : 


Pg  =  1  -  T^[2N-l,N-l, 


NA2\ 

2<r2  )■ 


(38) 


II. 3  :  Envelope  Detector 

The  PDF  of  the  envelope  ( R )  for  a  single  sample  was  found  in  (8).  Hence,  for  a  given  Pfa,  the  detection 
threshold  is, 

7  =  7-20-2  ln  pFA .  (39) 

The  calculation  of  threshold  with  N  observation  samples  can  be  shown  to  be  [12], 

7  ~  TfH)  +  NyJl.  where’  (40a^ 

T  is  found  approximately  from, 

PFA  =  0.5(1  -  <j>~l{T))  (406) 

and  <£(•)  denotes  the  error  function.  More  details  for  the  Envelope  Detector  case  can  be  found  in  [12]. 

It  may  be  noted  that  unlike  the  square  law  and  envelope  detection  threshold  calculations  for  conventional 
radars  [13],  the  discretized  schemes  presented  here  do  not  use  matched  filtered  output  but  use  the  sampled  data 
directly. 
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Section  -  2.6  :  PIPELINED-ADAPTIVE  TRACKING  OF  MULTIPLE  SINUSOIDAL  FREQUENCIES 
Summary 


New  Pipelined-Adaptive  algorithms  are  proposed  for  tracking  multiple  Frequencies  or  Angles-of-Arrival 
(AOA)  of  moving  targets.  Pipelining  of  adaptive  filters  pose  a  critical  challenge  because  of  the  timing  mismatch 
arising  from  the  feedback  signals.  In  this  paper,  some  relaxation  techniques  [9]  will  be  utilized  to  pipeline  adap¬ 
tive  algorithms  for  high-speed  tracking  of  frequency/ AO  As.  Two  adaptive  tracking  algorithms  have  been  mapped 
into  pipelined  forms,  namely  Least-Mean  Squares  (LMS)  [3]  and  Recursive  Least-Squares  (RLS).  Preliminary 
simulation  studies  with  multiple  sources  indicate  encouraging  results. 

I.  Introduction  :  Pipelined  data-adaptive  algorithms  are  presented  for  passive  high-speed  tracking  of  multiple 
targets.  In  non-stationary  environment  or  when  target  locations  change  with  time,  block-mode  processing  of 
observation  data  is  inappropriate  while  adaptive  algorithms  are  more  preferable.  Various  adaptive  algorithms 
addressing  this  problem  exist  [3,5]  but  the  throughput  rate  of  these  algorithms  are  limited  by  usually  long  critical 
paths  of  the  adaptive  filters.  Critical  paths  can  be  reduced  by  pipelining  which  is  usually  accomplished  by 
introducing  appropriate  latches  at  intermediate  stages  to  divide  the  critical  path  into  multiple  disjoint  sections. 
Pipelining  allows  higher  sampling  rate  and  throughput  essential  in  many  radar  applications  such  as  in  digital 
microwave  receivers  [11].  However,  pipelining  of  adaptive  filters  pose  additional  challenge  due  to  timing  mismatch 
produced  by  feedback  signals  [9].  Recently,  some  relaxation  techniques  have  been  found  to  be  effective  in  pipelining 
certain  adaptive  algorithms  for  coding  and  communication  applications  [9].  In  this  work,  we  study  the  effectiveness 
of  relaxations  for  pipelining  adaptive  tracking  algorithms. 

It  may  be  noted  that  various  adaptive  algorithms  for  tracking  multiple  targets  do  exist,  including  LMS  [3], 
gradient  adaptive  lattice  (GAL)  [3],  least  squares  lattice  (LSL)  [3],  Recursive  Least  Squares  (RLS)  and  QR-based 
adaptive  tracking  algorithms  [5].  However,  to  the  best  of  our  knowledge,  none  of  these  adaptive  frequency  tracking 
algorithms  have  been  implemented  or  studied  in  pipelined  forms.  Here  we  present  the  results  of  relaxation-based 
pipelining  on  LMS  and  RLS  based  tracking  algorithms.  Research  on  pipelining  of  the  other  tracking  algorithms  is 
being  conducted  and  will  be  reported  later.  It  may  be  emphasized  here  that  pipelining  will  not  only  be  beneficial 
for  speeding  up  adaptive  tracking,  recent  studies  indicate  that  pipelining  can  be  also  effective  for  reduction  of 
both  power  consumption  [1]  and  chip-area  [8]  using  appropriate  folding  techniques. 

II.  Look-Ahead  Pipelining  (LAP)  :  Consider  the  first  order  recursive  equation  given  by, 

y(n  +  1)  =  ay(n)  +  x(n).  (1) 

The  corresponding  transfer  function  is  given  by, 

H(z)  =  (16) 

1  —  az  1  v  ' 

By  applying  an  M-step  look-ahead  using  back-substitution, 

M- 1 

y(n)  =  aMy(n  —  M)  +  alx(n  —  i  —  1).  (2) 

*=o 

It  can  be  easily  shown  that  the  transfer  function  corresponding  to  both  (1)  and  (2)  are  identical.  Note  that  y(n ) 
no  longer  depends  on  the  previous  output  sample  y(n  ~~  1)  but  on  an  output  that  is  M  samples  back  in  time,  Le., 
y(n  —  M).  Hence,  the  immediate  dependence  problem  has  been  removed  [4,  6],  z.e.,  the  signals  can  be  sampled 
more  often  or  there  is  more  time  for  computation.  This  implies  that  the  throughput  of  the  logic  unit  is  increased 


70 


by  a  factor  of  M  leading  to  high-speed  implementation.  This  technique  is  referred  to  as  time-domain  [6,  10] 
or  Clustered  look-ahead  pipelining  [7]  Note  also  that  speed-up  in  throughput  is  achieved  at  the  cost  of  higher 
hardware  complexity  due  to  the  overhead  term  in  (2). 

III.  Relaxation  Techniques  :  The  above  LAP  scheme  maintains  exact  equivalence  in  the  transfer  function. 
However,  the  second  term  in  (2)  is  an  overhead  term  and  from  the  standpoint  of  hardware,  exact  computation  of 
this  overhead  term  may  be  impractical,  particularly  for  real-time  adaptive  implementations.  It  has  been  shown  in 
[9]  for  a  variety  of  coding  and  communication  related  applications,  that  these  overhead  terms  can  be  approximated 
under  certain  circumstances. 

III.l.  Sum  Relaxation  :  In  (2),  if  a  &  1,  and  if  x(n)  remains  approximately  constant  over  M  clock  cycles,  then 
we  can  replace  the  overhead  term  in  y(n  -f  M)  by  [9], 

y(n  +  M)  —  aMy(n)  +  Mx(n)  (3) 


III. 2.  Product  Relaxation  :  When  a  is  time- varying  (represented  more  appropriately  by  a(n)),  but  its  magnitude 
is  close  to  unity,  then  a(n)  can  be  written  as  (1  -  a'(n))  where  a'(n)  is  close  to  zero.  The  equation  for  y(n  +  M) 
can  be  approximated  as  [9] , 

M  —  1 

y(n  +  M)  =  (1  —  Ma\n))y{n)  +  ^  ax(n)x{n  +  M  —  1  —  i).  (4) 

»=o 


IV.  Adaptive  Frequency  Tracking  using  Pipelined  LMS  Adaptive  Filters  :  Consider  the  un-pipelined 
LMS  algorithm  which  is  referred  to  as  the  ‘serial9  LMS  (SLMS)  algorithm 


K(n)  =  (1  -  aLMS)^o(n  -  !)  +  «LMSu2(n) 
<*.) = 

pEi{n) 

W(n)  =  W(n  -  1)  +  /i(n)e(n)U(n) 
e.(n\  =  rffn'i  -  WTfn  -  l'lUCn'l 


(5) 

(6) 

(7) 

(8) 


where,  0  <  ^  ^  ^iaS  ^6en  use<*’  P  1S  or(^er  °f  transversal  filter  and  E^(n)  is  the  power  of  the  signal 

samples  within  the  tracking  window  [3],  By  applying  M- step  look-ahead  to  equations  (5)  and  (7),  we  have 


K{n)  =  (1  -  aLMS)M^(n  -  M) 
aLMS  H  _  aLMS)‘u2(n  _  *)-(9) 


M- 1 

E1 

»=o 


M- 1 

W(n)  =  W(n  -  M)  +  /i(n)  ^  e(n  -  i)U(n  -  i). 

1=0 


(10) 


This  introduces  M  latches  in  the  recursive  loops  which  may  be  redistributed  to  pipeline  the  feedback  multiply-add 
operation  by  M  levels.  By  substituting  the  equation  (10)  into  equation  (8)  we  obtain  the  error  equation 


e(n)  =  d(n )  — 


M—  1 

^(n)  e(n  —  i  —  l)U(n  —  i  —  1) 
*=o 


T 

U(n)  -  WT(n  -  M  -  l)U(n). 


(11) 


Clearly,  the  number  of  overhead  terms  after  applying  look-ahead  is  rather  high.  By  applying  the  sum  relaxation 
to  equation  (11)  and  replacing  W(n  —  M  —  1)  by  W(rz  —  M)  we  can  approximate  equation  (11)  as, 


e(n)  =  d(n)  —  WT(n  —  M)U(n). 


(12) 
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The  sum  relaxation  is  applied  assuming  that  /i(n)  is  relatively  small  and  hence  the  second  term  in  equation  (11) 
does  not  have  a  dominating  effect  on  the  error  calculation.  We  also  approximate  equation  (9)  to 


Kin)  =  (!  _Q!lms)m 


M  —  l 


Ee0{n  -  M)  +  <*LMS  X)  u2(n  "  0 

t=0 


(13) 


Equations  (6),  (10),  (12)  and  (13)  constitute  the  relaxed  look-ahead  pipelined  LMS  algorithm  (PLMS).  It  may 
be  noted  here  that  the  PLMS  version  given  in  [9]  does  not  make  use  of  or  pipeline  this  adaptive  error  calculation 
given  in  (13)  which  has  been  shown  to  be  convenient  in  adaptive  tracking  [3]. 

Note  that  the  hardware  complexity  after  relaxed  look-ahead  pipelining  has  increased  by  (N  -f  1  ){M  —  1)  adders 
because  of  the  overhead  terms  in  (9)  and  (10).  The  architectures  of  the  SLMS  and  PLMS  filters  are  as  shown 
Fig.  la  and  Fig.  lb,  respectively.  By  comparing  the  critical  paths  of  the  two  architectures  we  see  that  by  proper 
distribution  of  the  extra  delays  introduced  by  pipelining,  the  pipelined  architecture  can  be  made  to  operate 
approximately  M  times  faster. 


V.  Adaptive  Frequency  Tracking  using  Pipelined  Recursive  Least- Squares  Algorithm  :  The  ‘serial’ 
recursive  least-squares  (SRLS)  algorithm  is  described  by  [2,9]  : 

w  _  A~1P(n-  l)u(n) 

U  1  +  A_1uT(n)P(n  —  l)u(rc) 
a(n)  =  d(n )  —  WT(n  —  l)u(n); 

W(n)  =  W(n  —  1)  +  k(n)a(n); 

P(n)  =  A"1  [P (n  —  1)  —  k(n)uT(n)P(n  —  1)]  ; 

where,  uT(n)  =  [w(n),  u(n  —  1),  ■  *  • ,  u(n  —  N  +  1)]  is  the  input  vector,  WT(n)  =  [iui(n),  •  •  • ,  u>jv(n)]  is  the  vector 
of  weights,  d(n)  is  the  desired  signal,  k(n)  is  the  Kalman  filter  gain,  a(n)  is  the  error  and  P(n)  =  ^>"~1(n),  <fi(n) 
being  the  deterministic  autocorrelation  matrix  of  the  input  signal.  <j)(n)  A  =  An-tu(z)uT(z).  This  RLS 

algorithm  solves  for  W(n)  in  the  following  normal  equations, 

0(n)W(n)  =  0(n) 

where,  the  cross-correlation  vector  0(n)  A  Y^=o  An_,d(i)u(i).  0(n)  and  <j>(n)  can  be  computed  recursively  as, 


0{n)  —  A  0(n  —  1)  +  d(n)u(n); 

<f>(n)  =  A  <f>(n  —  1)  +  u(n)uT(n) 

Relaxed  look-ahead  pipelining  is  applied  to  the  above  two  recursive  equations  to  obtain 

0(n)  =  A  0(n  —  M)  +  L^d(n)u(n); 

<j)(n)  =  A  (j)(n  —  M )  +  L^u(n)uT(?i); 

where,  La  is  the  look-ahead  factor.  The  sum  and  product  relaxations  were  used  to  obtain  the  pipelined  equations. 
Using  these  equations  to  solve  for  W(n),  the  pipelined  RLS  (PRLS)  equations  can  be  re-derived  [9]  and  are  given 
as, 

A -'Pjn-MMn) 

1  +  X~xuT(n)P(n  —  M)u(n) 
a(n)  =  d(n)  —  WT(n  —  M)u(n) 

W(n)  =  W(n  -  M)  +  k(n)a(n) 

P(n)  =  A""1  [P(n  —  M)  —  k(n)uT(n)P(n  —  M)] 
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The  architecture  for  both  SRLS  and  PRLS  are  given  in  Fig.  2a  and  Fig.  2b,  respectively.  We  can  see  that  the 
hardware  complexity  is  almost  the  same  as  that  of  the  serial  algorithm  except  for  an  additional  2 M  latches.  One 
set  of  M  latches  corresponds  to  that  required  to  pipeline  the  W- loop  and  the  other  to  pipeline  the  P- loop.  The 
M  latches  can  be  redistributed  within  the  architecture  so  as  to  maximize  throughput.  Furthermore,  by  employing 
the  folding  transformation  [8],  the  hardware  of  the  pipelined  algorithm  can  be  further  reduced. 

VI.  Simulation  Results  :  Several  simulations  have  been  conducted  to  verify  the  performance  of  the  pipelined 
adaptive  algorithm  in  tracking  time  varying  frequencies  at  various  noise  levels. 

Simulation  1  :  The  data  set  consists  of  2  real  sinusoids  of  0.4375  Hz  and  0.1250  Hz  which  undergo  a  step  change 
to  0.3750  Hz  and  0.0625  Hz  respectively.  The  signals  are  at  signal  to  noise  ratios  (SNRs)  of  20  dB  and  15  dB 
respectively.  Fig.  3a  and  3b  show  the  tracking  characteristics  of  the  SLMS  (when  M  =  1)  and  PLMS  (with  M 
=  3),  respectively,  keeping  the  new  parameter  at  a  =  0.04.  Fig.  4a  and  4b  show  the  corresponding  results  using 
SRLS  (with  M  =  1)  and  PRLS  (with  M  =  3),  respectively,  with  A  =  0.95.  Clearly,  in  both  cases  convergence  for 
the  target  with  higher  SNR  is  quicker  than  the  low  SNR  target.  Furthermore,  there  is  little  effect  on  convergence 
time  due  to  relaxations  used  in  pipelining  especially  in  the  case  of  the  PLMS.  In  case  of  PRLS,  there  appears  to 
be  some  jitter  when  trying  to  keep  up  with  the  change  for  the  lower  SNR  target. 

Simulation  2  :  The  effectiveness  of  PLMS  and  PRLS  in  tracking  time-varying  frequencies  has  also  been 

tested  by  letting  the  algorithm  track  two  sinusoidal  FM  signals  (/ci  =  0.3750 Hz,  fc 2  =  0.1250i7z  and 
A/i  =  A/2  =  0.0625 Hz),  where  fc  and  A /  represent  center  frequency  and  peak  frequency  deviation  of  an 
FM  signal,  respectively.  Both  signals  are  at  20  dB.  The  simulations  are  shown  in  Fig.  5a  and  5b  for  the  SLMS 
case  ( M  =  1)  and  the  PLMS  case  (M  =  3)  respectively  with  a  =  0.15.  Fig.  6a  and  6b  show  the  simulation 
results  for  the  SRLS  ( M  =1)  and  PRLS  (A/=3)  cases  respectively  with  A  =  0.7.  Again  the  PLMS  and  the  PRLS 
show  minimal  convergence  degradation. 
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Figure  6:  Adaptive  tracking  using  (a)  SRLS  (b)  PRLS  Figure  4:  (a)  SRLS  adaptive  Tracking  (b)  PRLS  adaptive  tracking 
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CHAPTER  3 


SYSTEM  IDENTIFICATION  AND  HARWARE  IMPLEMENTATION  PROBLEMS 
Introduction 

The  rational  System  Identification  theory  is  closely  related  to  the  receiver  design  problem.  In  particular, 
Angle-of- Arrival  (AOA)  and  frequency  estimation  are  two  of  the  most  important  integral  parts  of  most  radar 
receivers  but  these  two  problems  can  be  addressed  as  special  cases  of  rational  system  identification  problems. 
Furthermore,  digital  EW  receivers  would  require  many  digital  filters  for  various  purposes  such  as,  anti-aliasing, 
image  suppression,  IF  and  etc..  Synthesis  of  digital  HR  filters  from  any  arbitrary  frequency  domain  specifications 
is  also  one  of  the  important  problems  addressed  by  the  proposed  system  identification  framework.  Identification 
of  unknown  discrete-time  linear  systems  is  a  fundamental  problem  in  signal  processing.  Among  many  available 
parametric  models,  pole-zero  or  rational  transfer  function  model  is  one  of  the  most  effective  and  practical  rep¬ 
resentations.  Optimal  estimation  of  rational  model  parameters  will  be  the  focus  of  this  part  of  the  report.  The 
system  identification  and  signal  analysis  problems  considered  here  are  fundamental  in  nature  and  the  results  are 
expected  to  have  impact  and  usefulness  in  a  wide  range  of  applications  including  EW  receiver  design. 

Applications  of  System  Identification  abound  in  Communication  systems,  Automatic  Control  systems, 
Aerospace  and  Mechanical  Systems,  Econometrics  and  many  other  fields.  Digital  filter  design  from  frequency 
and/or  time-domain  information  has  extensive  applications  in  speech  or  image  processing,  communication,  radar 
or  sonar  signal  processing,  bio-medical  signal  processing,  Digital  Instrumentation  and  Control  and  in  various 
other  fields.  Depending  on  the  application,  the  design  specifications  of  an  unknown  system  may  be  available 
or  prescribed  in  the  time-domain  (T-D)  as,  (i)  Impulse  Response  (IR)  or  (ii)  Input-Output  (10)  data,  and  in 
the  frequency-domain  (F-D)  as  (iii)  Frequency  Response  (FR)  data.  The  standard  synthesis  or  identification 
problem  is  to  estimate  the  numerator  and  denominator  polynomial  coefficients  that  match  the  prescribed  specs 
in  the  least-squares  (LS)  sense.  It  is  well-known  that  these  LS  problems  are  highly  non-linear.  Some  existing 
approaches  minimize  ‘equation  errors’  instead  of  the  true  fitting  errors  and  others  modify  or  linearize  the  true 
model-fitting  criteria  for  iterative  estimation  of  the  numerator  and  denominators  simultaneously . 

The  main  goal  in  this  part  of  the  work  is  to  exploit  certain  powerful  theoretical  results  in  Numerical  Analysis 
to  theoretically  decouple  the  multidimensional  nonlinear  criteria,  into  two  distinct  problems  :  (1)  a  purely  linear 
problem  for  estimating  the  numerator  and  (2)  a  non-linear  problem  for  estimating  the  denominator.  The  nonlinear 
part  is  then  reparameterized  by  invoking  results  on  projection  operators.  In  this  form,  the  denominator  criterion 
possesses  a  weighted  matrix  structure  which  is  convenient  for  iterative  optimization.  But  more  importantly, 
once  the  optimal  denominator  is  known,  the  optimal  numerator  is  found  with  only  a  single  step  of  linear  LS 
estimation.  Removal  of  the  numerator  estimation  from  the  iterative  process  reduces  computational  complexity 
when  compared  with  existing  simultaneous  estimators  in. 

The  theoretical  results  as  well  as  the  algorithmic  framework  we  propose  here  encompass  a  comprehensive 
class  of  system  identification  problems  in  time  and  frequency  domains.  This  important  underlying  common  theme 
appears  to  have  remained  unrecognized  and  un-utilized.  In  fact,  one  of  our  goal  is  to  establish  the  analogies  and 
equivalences  between  the  time-domain  and  frequency-domain  optimization  approaches  which  seem  to  have  evolved 
independently.  Our  hope  is  that  a  thorough  study  and  proper  understanding  of  these  equivalences  might  enable 
us  to  apply  and  exchange  useful  ideas  from  one  domain  to  the  other.  It  may  also  lead  to  combined  optimization 
in  the  frequency  and  time  domains  by  matching  the  desired  characteristics  in  both  domains  simultaneously. 

The  proposed  unified  framework  is  expected  to  provide  intuitive  and  useful  theoretical  insights  into  various 
time-domain  and  frequency- domain  identification  and  synthesis  problems.  For  example,  the  1-D  SISO  algorithms 
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can  be  extended  to  multi-dimensional  (m-D)  and  multi-input/multi-output  (MIMO)  problems  in  a  straight¬ 
forward  manner. 

Look-ahead  pipelining  has  been  found  to  be  very  effective  for  attaining  high  sampling  rate  and  high  compu¬ 
tation  speed  in  low-cost  VLSI  implementation  of  recursive  digital  filters.  The  well-known  Scattered  Look-ahead 
implementation  of  Recursive  IIR  filters  achieves  stability  at  the  cost  of  increased  multiplication  and  latch  complex¬ 
ities  and  considerable  delay  in  output  generation.  The  clustered  look-ahead  approach  can  not  always  guarantee 
stability  [1].  We  present  a  new  scheme  (referred  to  a s  distributed  look-ahead)  which  is  a  compromise  between 
the  two  existing  look-ahead  approaches.  The  proposed  scheme  appears  to  avoid  some  of  the  potential  drawbacks 
in  various  pipelined  implementations  of  recursive  filters.  Our  work  shows  that,  in  order  to  attain  stability,  the 
output  samples  need  not  be  clustered  or  equally  scattered.  Indeed,  in  many  filter  design  problems,  stability  can 
be  maintained  by  using  unequally  distributed  past  output  samples.  When  compared  with  the  scattered  approach, 
the  proposed  scheme  uses  fewer  number  of  pole-zero  cancelations  and  the  introduced  roots  are  not  necessarily  at 
the  same  radii  as  the  original  filter  poles.  Hence,  the  proposed  distributed  look-ahead  scheme  has  reduced  multi¬ 
plication  and  latch  complexities,  higher  area-efficiency  and  it  produces  outputs  with  reduced  delay.  The  proposed 
DLA  scheme  has  been  used  for  high-speed  implementation  of  both  1-D  and  2-D  Recursive  Digital  Filters. 

The  look-ahead  pipelined  recursive  filters  discussed  above  are  obtained  primarily  via  transformation  of  a 
given  un-pipelined  transfer  function.  For  these  approaches,  it  is  assumed  that  the  un-pipelined  transfer  function 
has  already  been  designed  as  an  intermediate  step.  In  this  project,  we  also  present  a  new  algorithm  (OM-LA) 
for  direct  and  optimal  estimation  of  the  coefficients  of  recursive  filters  in  look-ahead  pipelined  form.  OM-LA  is 
developed  by  appropriate  modification  of  a  recently  proposed  optimal  method  (OM)  for  designing  un-pipelined 
filters  (developed  previously  by  the  PI  as  part  of  a  project  supported  by  the  AFOSR).  It  is  demonstrated  that  the 
proposed  one-step  approximation  can  achieve  superior  match  with  reduced  pipelined  filter  order  because  it  does 
not  rely  on  pole-zero  cancelations  as  in  current  LA  pipelining  approaches.  It  is  also  shown  that  the  denominator 
polynomial  can  be  constrained  to  possess  any  of  the  possible  look-ahead  configurations.  Unlike  some  existing 
methods,  OM-LA  minimizes  the  true  time-domain  fitting  error-norm  between  the  prescribed  and  the  estimated 
impulse  response  and  produces  superior  results. 
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Section  -  3.1  :  IDENTIFICATION  OF  1-D  RATIONAL  SYSTEMS  FROM  INPUT-OUTPUT  DATA 
Summary 


A  theoretical  and  algorithmic  framework  is  proposed  for  optimal  identification  of  rational  transfer  function 
parameters  of  discrete-time  linear  systems  from  Input-Output  (10)  data.  The  nonlinear  criterion  is  theoretically 
decoupled  into  a  purely  linear  problem  for  estimating  the  optimal  numerator  and  a  nonlinear  problem  for  the 
optimal  denominator.  The  proposed  decoupled  approach  has  reduced  computational  requirements  when  compared 
to  existing  algorithms  which  estimate  the  parameters  simultaneously. 

I.  INTRODUCTION  :  Identification  of  unknown  Linear  Time-Invariant  Discrete-Time  systems  is  a  critical  prob¬ 
lem  in  signal  processing  and  control  theory  [1-13,  15-22].  This  work  addresses  the  problem  of  optimal  identification 
of  the  parameters  of  rational  transfer  functions  by  Least-Squares  (LS)  fitting  of  observed  input-output  sequences. 
Optimization  of  the  LS  criterion  for  this  problem  requires  multi-dimensional  nonlinear  optimization  [1,  2,  15-21]. 
Many  existing  algorithms  either  modify  or  linearize  the  true  nonlinear  error  criterion  to  estimate  the  unknown 
parameters  simultaneously.  This  work  will  demonstrate  that  the  optimal  rational  model  identification  problem 
belongs  to  a  special  class  of  mired-nonlinear  optimization  framework  where  the  linear  and  nonlinear  variables 
separate  [14].  The  true  nonlinear  criterion  will  be  theoretically  decoupled  into  : 

(i)  a  purely  linear  problem  for  obtaining  the  optimal  numerators  and 

(ii)  a  nonlinear  problem  of  reduced  dimensionality  for  determining  the  optimal  denominators. 

The  decoupled  criteria  retain  the  global  optima  of  the  original  criterion.  Only  the  criterion  for  the  denominator 
is  nonlinear  but  it  possesses  a  weighted-matrix  structure  which  is  utilized  for  minimizing  it  iteratively.  The 
optimal  numerator  is  estimated  in  one  step.  Hence,  unlike  some  existing  algorithms  which  estimate  both  sets  of 
parameters  iteratively  [2],  the  proposed  computational  algorithm  has  reduced  computational  requirements. 


II.  PROBLEM  Formulation  :  Rational  transfer  function  representations  of  a  SISO  plant, 

_  q(Q)  +  a(l)*-1  +  ---  +  q(g)z~'?  A{z)_ 

K)~  l  +  b(l)z-i  +  ---  +  b(p)z~p  =  B{zy 

=  h{ 0)  +  h(l)z~l  +  ■  ■  ■  +  h(N  -  1  JxrC*-1)  +  •  •  • , 


(1) 

(2) 


where,  6(0)  =  1.  Fig.  1  depicts  what  is  commonly  known  as  the  output-error  model  of  a  plant,  where,  y0(n )  and 
y(n)  denote  the  true  and  observed  (possibly  noisy)  output  signals,  respectively,  and  i>(n)  denotes  the  observation 
or  measurement  noise.  Let, 

x  A  [#(0)  #(1)  •••  x(N  —  1)]T  (3a) 

and 

y  A  [2,(0)  2,(1)  •••  y(N  —  1)]T  (36) 

denote  the  vectors  containing  the  N  input  and  observed  samples,  respectively.  In  vector  form,  the  unknown  model 
parameters  are  defined  as, 

a  A  [q(0)  q(l)  •••  a(q)f  (4q) 

and 

b  A  [1  6(1)  •  •  •  6(p)]T .  (46) 

The  problem  under  consideration  in  this  part  of  the  project  can  be  stated  as  follows  : 


78 


Given  the  observed  output  data  y  and  the  input  data  x,  estimate  the  optimal  model  parameters  a  and  b  by 
minimizing  the  following  LS  model-fitting  criterion  : 


"'I  Y.  »(• 


Regarding  methods  related  to  this  work,  Kalman  [1]  had  defined  an  equation  error  to  solve  this  problem  (KM), 
whereas  Steiglitz  and  McBride  (SMM)  iteratively  minimized  a  modified  error  criterion  [2]  to  estimate  both  sets 
of  parameters  simultaneously. 

III.  PROPOSED  Method  (OM-IO)  :  Let  Hb(z)  be  the  inverse  filter  corresponding  to  B(z ),  i.e., 

B(z)Hb(z)  =  1.  (6) 

This  is  a  convolution  operation  and  hence,  in  matrix  notations, 

BbUb  =  I*,  (7) 

where,  In  denotes  an  N  x  N  identity  matrix;  Bj  and  Hb  are  convolution  matrices  formed  as, 

Bb(i,j)  Ab(i-j),  (8a) 


H b(i,j)  A  Hb(i-j),  for ,i,j  = 
Note  that  both  these  matrices  are  lower-triangular.  In  partitioned  form, 


H>  =  [H,|Hr 


where,  Bu  €  IRArx(?+1),  B  G  H(  G  jR^xfo+i)  and  Hr  £  ^NxiN-q-i)  Using  ^  and 

that  the  input  is  causal,  Le.,  x(n)  =  0,  for  n  <  0,  the  optimization  criterion  in  (5)  can  be  restated  as, 


min  y(i)  —  x(i)  *  hb(i)  *  a(i)J  , 


where,  *  denotes  the  convolution  operation.  In  matrix  notations  the  problem  is  equivalent  to  : 


min||e(a,  b)||2  A  min||y  —  XH/a|| 

a,b  =  a.b  f 


where, 


X(i,j)  A  i(i-j)  for,  i,j  =  (116) 

This  is  a  mixed  optimization  problem  where  the  linear  and  nonlinear  variables  appear  separately .  If  H /  ( i.e b) 
is  known,  then  the  linear  LS  estimate  of  the  numerator, 


a  A  (XH,)#y,  (12) 

where,  (XHj)#  A  ((XH;)t(XHj))_1(XH|)t  denotes  the  pseudo-inverse  of  (XHj).  Plugging  a  back  in  (11a), 
the  optimization  criterion  for  b  is  given  by, 

min||e(b)||2  A  min  ||y  -  PXH,y||2  =  min  ||[IW  -  PxH,]y||2  (13) 

b  =  b  b 
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where,  Pxh,  A  XH/((XHj)t(XH/))~1(XH/)t,  denotes  the  projection  matrix  of  (XHj).  In  (13),  the  parameters 
in  b  are  indirectly  related  to  the  error  criterion  in  a  rather  complicated  manner  through  Pxh,  ■  Next,  the  inherent 
matrix-structure  of  the  criterion  in  (13)  is  utilized  to  reparameterize  the  error  criterion  by  relating  it  directly  to 
the  coefficients  in  b. 

Let  Xi(z)  be  the  inverse  of  the  input  sequence  X(z ),  i.e.,  X(z)X/(z)  =  1.  Similar  to  (6)  and  (7),  in  matrix 
notation, 

X/X  =  INl  (14a) 

where,  X/  e  JRJVxAr  is  also  a  lower  triangular  matrix  defined  as, 


X/  A  Xf(i-j),  for,  iyj  = 


(14  b) 


For  finite  N,  this  inverse  exists  as  long  as  the  first  element  of  the  input  sequence  is  non-zero,  i.e.,  x(0 )  ^  0.  This 
is  not  a  major  restriction  for  the  causal  systems  under  consideration  in  this  work  because  the  output  will  have 
non-zero  leading  values  only  when  there  is  non-zero  input.  But  it  would  be  desirable  that  X(z)  be  minimum- 
phase,  otherwise  X/(z)  may  be  unbounded  for  some  values  of  z  which  in  turn  may  result  in  very  high  magnitudes 
of  xj(n)  for  large  N.  Combining  (7)  and  (14)  and  using  the  partitioned  forms  of  (9), 


BjX/XHj  =  lN 


X/X[H,|Hr] 


(15a) 


BjX/XH/ 

BjX/XH/ 

^J+i) 

0(4+l)x(Af-4-l) 

BtX/XH, 

|  BTX/XHr 

_0(JV-4-l)x(4+l) 

1  *"• 

1  L 

i 

1  1 
>  <&■ 

t  1 

5, 

1  wf' 

The  bottom-left  corner  element  shows  that  the  N  x  (N  —  q  —  1)  matrix  Xj B  and  the  N  x(q  +  l)  matrix  XH /  are 
orthogonal,  z.e.,  (BtXj)(XH/)  =  _i)x($+i)-  By  construction,  rank(X.J B)  +  rank(XMi)  =  N.  Hence, 

according  to  a  property  of  projection  matrices, 


Px^b  +  Pxh,  =  I  at. 

Using  this  result  in  (13),  the  following  reparametrized  optimization  criterion  is  obtained, 

min  ||Px?-By||2  =  min  ||X]’B(BTX,X],B)-1BTX/y||a 

b  1  b 

=  min  yTXjB(BTX/XjB)-1BTX/y. 

b 

In  order  to  obtain  an  expression  more  convenient  for  optimization,  define, 


It  can  be  easily  shown  that, 


z  A  X/y. 
Btz  A  Zb, 


where,  the  matrix  Z  is  constructed  with  the  elements  of  z  as, 


z(q  +  1) 

*(«) 

o 

0 

z{q  +  2) 

z(q  + 1) 

•••  *(1)  *(0)  ••• 

0 

z  A 

z(p) 

z(p-l) 

z(0) 

_z(N  -  1) 

z(N  -  2) 

1 

^3 

1 

1 — 1 

1 _ 

(16) 

(17a) 

(176) 

(18) 

(19a) 

(196) 
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Using  (19a)  into  (17),  the  optimization  criterion  can  be  re-written  as, 

min  bTZT(BTX/X/ B)_1Zb.  (20) 

b 

Equation  (12)  and  the  reparametrized  criterion  in  (20)  are  the  final  decoupled  forms.  It  should  be  emphasized 
here  that,  thus  far,  the  theoretical  derivations  are  mathematically  exact,  i.e.,  no  linearization,  approximation  or 
modification  have  been  introduced  at  the  outset. 

According  to  the  Theorem  stated  in  the  Appendix  [14],  if  b  is  estimated  by  minimizing  the  criterion  in  (20) 
and  if  that  estimate  is  utilized  for  computing  a  using  (12),  then  the  resulting  estimates  are  the  unique  and  global 
minimizers  of  the  original  criterion  in  (5)  or  (11).  Furthermore,  once  the  optimal  b  is  known,  the  estimation  of 
the  optimal  a  in  (12)  is  a  linear  problem.  But  more  importantly,  a  needs  be  computed  only  once. 

Algorithm  :  The  nonlinear  criterion  in  (20)  appears  to  be  a  weighted  quadratic  in  the  unknown  vector  b. 
But  the  weight  matrix  (BTX/XjB)"1  itself  is  dependent  on  the  unknowns  in  B.  The  computational  algorithm 
exploits  this  weighted  quadratic  structure  of  the  criterion.  At  k- th  iteration,  the  algorithm  minimizes, 

min  bT[ZT(BT(*-1)X/XfB^_1))-1Z]b,  (21) 

b 

where,  is  formed  by  using  the  estimate  of  b  obtained  at  the  previous  iteration.  b(°)  A  [1  0  •  *  -0]T  can 

be  used  as  the  initial  estimate  of  b  to  start  the  iterative  process.  Otherwise,  the  initial  estimates  could  also  be 
found  by  setting  the  middle  matrix  (BtXjXj’B)-1  to  identity,  i.e.,  by  optimizing, 

min  bTZTZb.  (22) 

b 

To  ensure  non-trivial  solutions,  6(0)  is  set  to  unity.  Once  the  iterations  converge,  the  estimated  b  is  used  in  (12) 
to  linearly  estimate  the  numerator  coefficient  vector  a. 

On  the  Relationships  with  Other  Methods  :  The  proposed  theoretical  and  algorithmic  framework  appears 
to  be  the  most  general  one  in  its  own  class  of  1-D  deterministic  rational  System  Identification  (SID)  algorithms. 
In  fact,  a  large  body  of  work  on  SID  can  be  formulated  as  special  cases  of  OM-IO.  For  example,  in  case  of  Impulse 
Response  (IR)  fitting,  i.e.,  when  x(n)  =  <$(n)  and  y{n)  A  hd{n),  the  desired  IR,  an  optimal  method  (OM)  has 
been  developed  recently  [8,9].  The  work  in  this  part  of  the  project  may  be  considered  to  be  a  further  generalization 
OM.  The  Evans-Fischl  Method  (EFM)  [5]  was  an  early  precursor  of  OM.  But  EFM  dealt  only  with  the  IR  fitting 
problem  and  it  is  applicable  only  for  the  strictly-proper  case,  i.e.,  when,  p  =  q  +  1.  Furthermore,  the  recently 
proposed  Maximum- Likelihood  Method  for  exponential  modeling  (known  as,  KiSS  or  IQML)  is  basically  a  complex 
version  of  EFM  with  conjugate-symmetry  constraints  imposed  on  the  B(z)  coefficients  [6,  7].  Hence,  KiSS/IQML 
is  also  an  important  special  case  of  OM-IO.  Furthermore,  when  p  =  q+ 1,  the  initialization  step  of  OM  is  identical 
to  Prony’s  Method  [10]  or  Covariance  Method  of  Linear  Prediction  [11,  13].  For  general  cases,  Shanks  [3]  and 
Burrus-Parks  [4]  also  estimated  the  denominator  using  the  initialization  step  of  OM.  For  numerator,  the  linear 
estimator  in  (12)  was  used  by  Shanks  whereas  Burrus-Parks  used,  a(k)  =  K^)^d(k  —  i),  for  k  =  0, 1, . . . ,  q. 

Finally,  the  formulation  presented  in  here  appears  to  be  quite  well-suited  for  deconvolution  [22].  Specifically,  if 
the  output  and  the  Channel  IR  (or,  alternately,  the  estimates  of  a  and  b)  are  available,  then  the  criterion  in  (11) 
can  be  appropriately  modified  to  obtain  an  LS  or  MLE  of  the  unknown  input  vector  x. 

IV  :  SIMULATION  Results  :  In  all  figures,  the  true  and  modeled  impulse  responses  are  shown  in  solid  and 
dotted  lines,  respectively. 

Simulation  1  :  In  this  case,  white  noise  was  passed  through  an  ARMA(7,3)  system  with  an  arbitrary  impulse 
response.  The  output  was  corrupted  with  uncorrelated  white  noise.  The  first  N  =  30  input  and  output  samples 
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were  collected  for  identifying  the  system.  True  model  orders  were  used  for  identifying  the  system.  The  results 
with  30dB  and  15dB  SNR  values  are  shown  in  Fig.  2A  and  2B,  respectively. 

Simulation  2  -  Model  Reduction  :  For  the  same  data  sets  of  Simulation-1  an  ARMA(5,3)  model  was  used  for 
identifying  the  system.  Note  that  the  denominator  order  is  less  than  the  true  order  in  this  case.  The  results  with 
30dB  and  15dB  SNR  values  are  shown  in  Fig.  3A  and  3B,  respectively. 

The  results  of  Simulation- 1  indicate  that  the  proposed  algorithm  is  able  to  match  the  unknown  model  impulse 
response  very  closely  by  minimizing  the  output  error  norm.  Simulation-2  demonstrates  that  the  algorithm  also 
has  the  capability  of  obtaining  reduced  order  models  with  good  fit. 

Number  of  Iterations  for  Convergence  and  CPU  Times  :  For  30dB  SNR,  the  number  of  iterations  for 
convergence  for  actual  and  reduced  order  cases  were  found  to  be  8  and  5,  respectively.  The  iterations  were 
terminated  in  both  cases  when  ||b;+i  —  bj||2  <  10“3  was  achieved  in  each  case.  The  corresponding  CPU  times 
on  VAX-8550  were  3.0  and  2.59  seconds,  respectively.  Similar  differences  in  performance  were  found  for  other 
SNR  values  also.  In  general,  the  algorithm  showed  rapid  convergence  in  all  simulations  performed.  But  if  the 
unknown  system  is  non-minimum  phase  or  if  the  SNR  in  the  output  data  is  too  low,  the  algorithm  may  converge 
to  a  suboptimum  or  the  estimates  may  oscillate.  In  order  to  guarantee  convergence,  the  proposed  iterative 
transformation  must  be  a  contraction  mapping.  This  may  be  difficult  to  demonstrate  in  general  for  any  arbitrary 
input-output  data  set.  Theoretical  analysis  of  the  convergence  properties  of  the  iterative  algorithm  needs  to  be 
performed. 
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APPENDIX  :  Optimality  Properties  of  the  Separate  Estimators 

THEOREM  -  (Adapted  from  Theorem  2.1  in  [14])  :  If  b  is  a  global  minimizer  of  ||e(b)||2  in  (13)  and  a  is 
estimated  using  that  b  as  in  (12),  i.e., 

aA(XH,)#y,  (A  A) 

where,  H /  is  formed  using  b,  then  ||e(a,  b)||2  is  a  global  minimizer  of  ||e(a,  b)||2  and  ||e(a,  b)||2  =  ||e(b)||2. 
Conversely,  if  (a,b)  is  a  global  minimizer  of  ||e(a,b)||2,  then  b  is  a  global  minimizer  of  ||e(b)||2  and 
||e(b)||2  =  ||e(a,b)||2.  Finally,  if  there  is  an  unique  a  among  all  possible  minimizing  pairs  of  ||e(a,  b)||2,  then  a 
must  satisfy  (A.l). 

PROOF  :  See  [14], 
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Section  -  3.2  :  IDENTIFICATION  OF  1-D  RATIONAL  SYSTEMS  IN  THE  FREQUENCY  DOMAIN 

Summary 

A  new  Frequency-Domain  (FD)  approach  is  presented  for  optimal  estimation  of  rational  transfer  functions 
coefficients.  The  proposed  method  seeks  to  match  any  arbitrarily-shaped  FD  specifications  in  the  Least-Squares 
(LS)  sense.  The  desired  specifications  may  be  arbitrarily  spaced  in  frequency.  The  design  is  performed  directly 
in  the  digital  domain  and  no  analog  to  digital  transformation  is  necessary.  The  proposed  method  makes  use  of 
the  inherent  mathematical  structure  in  this  rational  modeling  problem  to  theoretically  decouple  the  numerator 
and  denominator  estimation  problems  into  two  smaller  dimensional  problems.  The  denominator  criterion  is 
nonlinear  but  possesses  a  weighted-quadratic  structure  which  is  convenient  for  iterative  optimization.  The  optimal 
numerator  is  found  linearly  by  solving  a  set  of  simultaneous  equations.  The  decoupled  criteria  retain  the  global 
optimality  properties.  The  performance  of  the  algorithm  is  demonstrated  with  some  simulation  examples. 

I  :  Introduction 

Traditionally,  digital  filters  are  designed  by  performing  Impulse  Invariance  or  Bilinear  transformation  on  available 
analog  designs.  Classical  analog  designs  utilize  polynomial  approximations  to  match  standard  filter  shapes  such 
as,  Low-Pass,  High-Pass  etc.  [9,  10].  An  obvious  drawback  of  classical  analog  design  techniques  is  that  filters 
with  arbitrary  or  non-classical  specifications,  as  in  case  of  a  notch  filter,  can  not  be  obtained.  In  this  part  of 
the  report,  a  direct  method  for  frequency-domain  design  of  digital  HR  filters  is  proposed.  The  method  seeks  to 
match  a  desired  frequency  response  with  any  arbitrary  shape  by  minimizing  the  optimal  LS  fitting  error  criterion. 
The  LS  criterion  for  this  problem  involves  multi-dimensional  nonlinear  search  and  several  linearized  or  modified 
approaches  have  been  developed  [2,  3,  21,  31].  There  have  been  some  ad-hoc  attempts  on  designing  digital  filters 
with  special  shapes  [9,  12].  Frequency  domain  version  of  Prony’s  algorithm  has  also  been  presented  recently 
[14,  15,  19,  25].  But  it  appears  that  the  underlying  mathematical  structure  inherent  in  this  rational  modeling 
problem  have  not  been  fully  exploited.  In  this  work,  the  frequency-domain  least-squares  problem  is  formulated  by 
identifying  the  orthogonal  projection  space  which  is  shown  to  be  formed  entirely  by  the  denominator  parameters. 
The  optimal  denominator  is  estimated  by  minimizing  the  exact  projection  space  which  is  independent  of  the 
numerator  coefficients.  The  optimal  numerator  estimation  problem  turns  out  to  be  a  simple  linear  LS  problem. 

It  is  demonstrated  in  this  work  that  the  optimal  rational  identification  problem  in  the  frequency-domain 
belongs  to  a  special  class  of  mixed-nonlinear  optimization  framework  where  the  linear  and  nonlinear  variables 
separate  [13].  It  is  further  shown  that  the  true  nonlinear  criterion  can  be  decoupled  into  : 

(i)  a  purely  linear  problem  for  obtaining  the  optimal  numerator  coefficients  and 

(ii)  a  nonlinear  problem  of  reduced  dimensionality  for  determining  the  optimal  denominator  coefficients. 

This  important  underlying  theoretical  and  algorithmic  aspects  of  designing  digital  filters  in  frequency-domain, 
appears  to  have  remained  mostly  un-utilized.  After  decoupling,  the  denominator  criterion  possesses  a  convenient 
weighted-matrix  structure  which  is  then  utilized  to  develop  an  iterative  minimization  algorithm.  Once  the  de¬ 
nominator  is  estimated,  the  optimal  numerator  is  found  only  once  with  linear  LS.  The  decoupled  criteria  retain 
the  global  optima  of  the  original  criterion.  The  proposed  approach  is  closely  related  to  some  time-domain  results 
developed  recently  by  the  present  author  [8,  16,  17].  The  design  methodology  described  here  will  be  based  on 
matching  desired  Discrete-Time- Fourier-Transform  (DTFT)  values  which  may  be  arbitrarily  spaced  in  frequency. 
But  the  algorithm  can  be  easily  modified  if  the  desired  specifications  are  available  in  the  form  of  DFT  values. 

The  Section  is  arranged  as  follows  :  In  Subsection  II,  the  rational  transfer  model  is  defined  and  the  frequency- 
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domain  identification  problem  is  stated.  In  Subsection  III,  some  existing  methods  addressing  this  problem  are 
briefly  outlined.  The  details  of  the  proposed  decoupled  solution  is  presented  in  Subsection  IV.  Some  simulation 
examples  are  given  in  Subsection  V  to  demonstrate  the  performance  of  the  proposed  approach. 

II  :  The  Rational  Transfer  Function  Model  and  The  Frequency- domain  Design  Problem 


An  ARMA(p,  q)  digital  filter  can  be  modeled  as  : 


H M  _  _  a(0)  +  a(l)z  1  +  ---  +  a(q)z  «  A  N(z) 

Zf(z)  -  ^ MO*  -  -I  +  H1)*-1+- +((,,)*->  ■  i 


h  A  [ft(0)  h(  1)  •  •  •  h(N-  I )]T  , 

be  the  vector  with  the  first  N  significant  samples  of  H(z)  and 


a  A  [ao  a\  •  •  •  < 
b  A  [1  &i  •  •  •  bp 


q]T  and 


be  the  numerator  and  denominator  coefficient  vectors,  respectively. 

Let  Hd(z)  represent  the  desire*/ IIR  filter  which  needs  to  be  modeled  as  H(z)  in  (1).  Using  the  notations  of 
equation  (1),  let  Hd(uk),  N(wk)  and  D(uk)  be  defined  as  the  frequency  response  values  of  Hd(z)}  N(z)  and  D(z ), 
respectively,  at  z  =  e*Wk.  The  frequency-domain  identification  problem  can  be  stated  as  follows  : 

Given,  Hd(uk),  at  fc  =  0,l,2,-..,  JV  —  1 ,  the  desired  frequency  response  values  (possibly  arbitrarily  spaced), 
estimate  the  parameters  in  N(u>k)  and  D(u> *)  by  optimizing  the  following  LS  error  criterion  : 


min||ea,||2  A 

a  b  = 


min 

a,b 


JV-1 

E  |^(«o 

*=o 


N{Ui)  2 

D(u>i)  ' 


(3) 


III  :  Some  Existing  Frequency- Domain  Direct  Design  Methods 

The  problem  stated  in  (3)  is  a  nonlinear  optimization  problem  and  standard  nonlinear  optimization  schemes  can 
be  used  [7,  11].  But  these  generic  algorithms  are  known  to  be  sensitive  to  initial  choice  of  estimates  and  they 
do  not  specifically  make  use  of  the  unique  mathematical  structures  inherent  in  this  problem.  Some  linearized 
methods  that  specifically  address  the  design  problem  stated  in  (3),  have  also  been  proposed  [2,  3].  More  recently, 
a  decoupled  algorithm  that  utilizes  divided-differences  and  Newton- Raphson,  has  been  reported  in  [14,  28].  In 
order  to  motivate  the  proposed  algorithmic  framework,  brief  outlines  of  some  of  the  direct  FD  design  methods 
are  given  next. 

III.l  :  Levy’s  Method  (LM) 

The  following  criterion  was  proposed  by  Levy  [2]  as  a  frequency-domain  counterpart  of  Kalman’s  original  work 
in  the  time-domain  [1]  : 

Az* .  <2 

min  ||2  A  min  |D(wf )#</(«,•)  -  JV(w,-)|  .  (4) 

’  ~~  ’  t=0 

Note  that  the  original  error  criterion  in  (3)  is  modified  in  Levy’s  case.  Apart  from  the  obvious  advantage  of 
single-step  linear  solution,  this  algorithm  does  not  possess  any  other  optimality  properties.  It  may  also  be  noted 
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that  Kalman/Levy-  type  approaches  for  the  ARM  A  problem  are  closely  related  to  Levinson’s  work  on  the  all-pole 
problem  [18],  where  only  the  first  term  of  the  error  criterion  was  minimized.  The  AR  parameter  estimation  work 
is  further  related  to  Prony’s  method  [19]  and  Pade  Approximation  [20].  Similar  error  criterions  for  the  ARMA 
problem  have  been  later  rediscovered  [21]  and  analyzed  [22]. 


III. 2  :  Sanathanan-Koerner’s  Prefiltering  Method  (SKM) 


The  earliest  work  that  most  closely  approximates  the  true  LS  fitting-error  criterion,  appears  to  be  due  to 
Sanathanan  and  Koerner  [3].  Their  goal  was  to  improve  upon  Levy’s  work  which  did  not  really  attempt  to 
optimize  the  true  criterion  in  (3).  In  this  case,  an  initial  estimate  of  the  denominator  coefficients,  Z)(°)(u;o)  is  first 
obtained  by  minimizing  Levy’s  criterion  in  (4)  and  then  the  following  modified  fitting  error  criterion  is  optimized 
at  the  k- th  iteration  [3], 


N-l 

min||esK-||2  A  min  V 

a.b  =  a,b  f ' 

t=0 


Diui)Hd(fJi) 


N{uj)  ^ 


(5) 


where,  denotes  the  denominator  estimate  at  the  previous  iteration  which  is  used  as  a  prefilter  for 

obtaining  the  estimates  at  the  following  iteration  step.  Note  that,  (5)  closely  approximates  (3)  and  both  are 
identical  if,  D(u)i)  =  But  using  (5),  the  unknown  parameters  in  a  and  b  can  be  estimated  simultane¬ 

ously  by  solving  a  set  of  linear  equations.  A  time-domain  counterpart  of  Sanathanan-Koerner’s  method  was  later 
discovered  independently  by  Steiglitz  and  McBride  in  [4],  though  the  later  work  is  definitely  more  well-recognized 
in  Signal  Processing  and  System  Identification  literature  [9,  10,  23,  24]. 


III. 3  :  Kumaresan’s  Decoupled  Method  -  Generalized  (KM-G) 


The  Frequency- Domain  error  criterion  in  (3)  has  been  recently  decoupled  by  Kumaresan  in  [14,  15,  25,  28],  where 
divided- difference  matrices  [26]  have  been  utilized.  Similar  to  a  time-domain  decoupled  algorithm  due  to  Evans 
and  Fischl  (EFM)  [6],  this  approach  was  originally  proposed  for  strictly-proper  cases,  i.e.,  when,  p  =  q  -f  1. 
In  the  brief  outline  given  below,  appropriate  modifications  have  been  introduced  in  order  to  generalize  KM  for 
any  arbitrary  numerator  and  denominator  orders.  For  q~ th  order  numerator  and  p-th  order  denominator,  the 


decoupled  criterion  for  estimating  the  optimal  denominator  is  : 

minhjffC'ff(CCff)_1Ch^  (6) 

b 

where, 

KA  [Hd(u0)  Hd(w i)  •••  (7a) 

denotes  the  vector  containing  the  N  samples  of  the  prescribed  frequency  response  data, 

C  A  B^UD  (76) 

’6(p)  •••  6(1)  1  .•  O' 

Bl  A  :  :  elR (N-i-iMN+p-t-1)  (7c) 
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Defining  f  A  UDh^  G  IR/7V+p  ?  1)xiV)  the  error  criterion  can  be  written  in  the  following  weighted-quadratic 
form  : 

Tj^H  { c'r'H  >j-i  (gj 


minbJ  Fii(CCii )"1Fb, 

b 

where,  F  E  jgJV-j-lxp+i  -s  formecj  usjng  the  elements  of  f  as  follows  : 
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The  optimal  denominator  coefficients  are  obtained  using  an  iterative  algorithm.  Once  the  optimal  denominator 
is  available,  the  numerator  is  estimated  as  : 

a  =  (D5U?+1)#h2, 


(10) 


where,  #  denotes  the  pseudo-inverse  and 
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It  can  be  easily  verified  that  for  the  special  case  of  p  =  q  + 1,  the  general  criteria  given  here  will  be  exactly  same 
as  the  one  given  in  [14,  28].  It  may  be  emphasized  here  that  the  frequency-domain  LS  algorithms  in  [2,  3]  are, 

(i)  Approximations  or  modifications  of  the  original  criterion  in  (3),  and 

(ii)  ( p  -1-  g)-dimensional  nonlinear  optimization  problems  for  estimating  a  and  b  simultaneously. 

In  contrast,  the  decoupled  method  (KM-G)  estimate  a  and  b  separately.  But  simulation  experiments  indicate  that 
the  desired  minimum  of  the  criterion  in  (8)  may  not  be  achieved  with  only  an  Evans-Fischl  type  LS  minimization 
of  (8).  Instead,  a  further  step  of  Newton-Raphson  had  to  be  incorporated  in  the  algorithm  in  order  to  achieve  the 
desired  optimum  [14].  Unlike  KM-G,  the  optimally  decoupled  method  developed  in  this  work  reaches  the  desired 
optimum  criterion  more  directly  and  without  using  Newton-Raphson. 

It  may  be  also  noted  that  Signal  Processing  Toolbox  of  the  widely  popular  MATLAB  software  package 
provides  a  direct  frequency-domain  design  macro  called  yulewalk,  which  basically  implements  a  modified  Yule- 
Walker  method  developed  by  Friedlander  and  Porat  [31].  This  method  does  not  attempt  to  minimize  the  true 
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criterion  in  (3).  Instead,  it  attempts  to  fit  the  deterministic  correlation  values  to  obtain  the  rational  model 
parameters  by  essentially  minimizing  an  equation  error.  The  simulation  section  includes  some  comparison  of  the 
performance  of  the  proposed  method  with  this  approach. 

IV  :  Proposed  Method  (OM-DTFT) 

For  time-domain  rational  model  identification  problems,  a  new  framework  has  been  recently  presented  for  decou¬ 
pling  the  denominator  and  numerator  problems  into  two  separate  but  lower- dimensional  optimization  problems 
[8,  16,  17].  In  this  Section  it  is  shown  that  the  nonlinear  frequency-domain  criterion  of  (3)  can  also  be  decoupled 
in  a  similar  fashion. 

Let  be  the  inverse  filter  corresponding  to  D(z ),  i.e., 

D(z)Hb(z)  =  1.  (13a) 

Clearly,  this  is  a  convolution  operation  in  time-domain  and  it  can  be  expressed  using  matrix  notation  as, 


DH&  =  Ijv, 


(136) 


where,  by  denotes  an  N  x  N  identity  matrix;  D  £  IR  V x A  and  Hj  €  IR/VXiV  are  defined  below  in  appropriate 
partitioned  forms  which  will  be  useful  in  the  algorithm  : 
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(14a) 


(146) 


where,  Bu  £  mNx{q+1\  B  £  ]RJV><(jV"?_1),  H,  £  IRiVx(?+1)  and  Hr  £  ntNx(JV-?-1).  If  the  vector  h,  defined  in 
(2a),  represents  the  finite  length  impulse  response  vector  containing  N  significant  Impulse  Response  values,  the 
frequency  response  at  any  frequency  w,-  will  be  given  as, 


TV— 1 

H(u>i)  A  H(z) \t=ej„.  =  h(n)e 

n= 0 


~3ojxn 


(15) 
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Stacking  the  model  frequency  response  values  at  all  the  N  specified  frequencies,  uq,  . . wjv-i,  the  model 
frequency-domain  vector  can  be  expressed  as  : 


A 


“1  e-juj° 
1  e" 


e-j(N- l)w0 


1  e~JU 


A  Wh. 


By  definition, 


H(z)  A  M  =  ff6(z)JV(z), 


using  (13a), 


where,  the  right-hand-side  represents  convolution  of  the  numerator  coefficients  with  the  inverse  sequence,  h^n) 
(corresponding  to  Hi>(z)).  Hence,  it  can  be  shown  that  the  model  impulse  response  vector  h  can  be  expressed  as, 


Using  this  in  (16), 


h  =  H/a. 


K  A  WH/a. 


With  these  definitions,  the  frequency-domain  filter  design  problem  in  (3)  can  be  restated  as, 

min||e||2  A  min  II -  WH,a||2.  (20) 

a,b  =  a,b 

Equation  (20)  is  an  exact  representation  of  the  original  criterion  in  (3),  albeit  in  the  vector-matrix  form.  This 
form  of  the  criterion  explicitly  demonstrates  the  linear  relationship  between  the  fitting  error  e  and  a  and  also  the 
nonlinear  relationship  between  e  and  b  through  the  matrix  H/.  From  this  equation,  it  is  also  apparent  that  this 
is  a  mixed  optimization  problem  where  the  linear  and  nonlinear  variables  appear  separately.  In  order  to  decouple 
the  numerator  and  denominator  estimation  problems,  consider  the  following.  If  H/  (i.e.,  b)  is  known,  then  the 
minimization  of  (20)  will  produce  the  linear  LS  estimate  of  a  as  follows, 

a  A  (WH,)#h£,  (21) 


where,  (WH/)#  A  ((WH/)T(WH/))-1(WH/)T.  In  practice  though,  b  needs  to  be  estimated  also.  Plugging  a 
back  in  (20),  the  optimization  criterion  for  b  is  found  as, 

min||h^-WH,a||2  =  min  ||hj  -  WH,(WH,)#h£|2 

a,b  b 

=  min||h^  -  PWH,h^||2 

=  min||(Ijv  -  PwH,)hf  ||2,  (22) 

where,  Pwh2  A  WH/((WH/)T(WH/))_1(WH/)T,  denotes  the  projection  matrix  of  (WH/).  Note  that  the 
numerator  and  denominator  estimation  problems  are  now  in  decoupled  forms  in  equations  (21)  and  (22),  respec¬ 
tively.  But  in  (22),  the  parameters  in  b  are  related  to  the  error  criterion  in  a  somewhat  complicated  manner 
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through  PwHi*  Interestingly,  the  operator  (In  -  Pwh,)  on  in  (22)  is  the  projection  component  in  that 
is  orthogonal  to  the  subspace  spanned  by  the  columns  of  WH j.  Next  it  is  shown  that  this  orthogonal  space  can 
be  completely  defined  by  the  denominator  coefficients. 

IV.  1  :  Reparameterization 


Let  W/  denote  the  inverse  of  the  DTFT  matrix  W,  Le.,  W/W  =  In-  This  inverse  exists  as  long  as  the 
frequencies  a/*’ s  are  distinct.  In  combination  with  (13b), 


DW/WHj  =  IN. 


Use  of  the  partitioned  forms  of  (14)  into  (23)  leads  to, 


'nr 

W/W  [H;|Hr]  = 

B„  W/WH, 

bJW/WH,  " 

— 

bt 

BtW/WH, 

|  BTW/WHr 

!(«+l)  j  %+l)x(N-q-l) 

.  0(at_?_i)x(3+1)  |  I(jv-?-i)x(jv-j-i). 


(23) 


(24) 


The  bottom-left  corner  element  shows  that  the  N  x  (N  —  q  —  1)  matrix  WjB  and  the  N  x  (q  +  1)  matrix  WH; 
are  orthogonal,  i.e.,  (BTW/)(WH;)  =  By  construction, 


rank(Wj  B)  +  rank(  WH,)  =  N. 
Hence,  using  a  property  of  projection  matrices, 


Pw/’b  +  Pwh,  =  In 


Using  this  result  in  (22), 


rnin  ||e6||2  A  min  HPwJ'b^II2 

=  min  ||WfB(BTW/Wj’B)-1BTW/h^||2 

b 

=  min  hfTWfB(BTW/WfB)-1BTW/hf. 

b 


(25a) 

(256) 


(26a) 

(266) 

(26c) 


Note  that  this  reparameterized  criterion  is  directly  related  to  b,  as  desired.  In  order  to  further  simplify  this 
expression,  define  a  vector  z  of  length  N  as, 

z  A  Wjhi  (27) 


such  that  the  criterion  in  (26)  becomes, 


min  zTB(BTW/WfB)-1BTz. 

b 


It  can  be  easily  shown  that, 


BTz  A  Zb, 


(28) 


(29a) 
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where,  the  matrix  Z  is  constructed  with  the  elements  of  z  as, 


'*(?+!)  *(?)  z(0)  0 

0 

Z  A 

z(p)  z(p- 1)  . 

z(l)  z(0) 

_z(N-  1)  z(N  -  2)  . 

•••  z(N-p- 1). 

Using  (29)  in  (28),  the  optimization  criterion  can  be  rewritten  as, 

min  bTZT(BTW/WjB)-1Zb.  (30) 

b 

Note  that  this  alternate  form  has  a  weighted-quadratic  structure  which  is  convenient  for  minimization.  Equations 
(21)  and  (30)  represent  the  final  decoupled  estimators  to  be  utilized  in  the  algorithm  described  below.  It  should 
be  emphasized  here  that,  thus  far,  the  theoretical  derivations  are  mathematically  exact,  i.e.f  no  linearization, 
approximation  or  modification  have  been  introduced  at  the  outset. 

Regarding  optimality  properties  of  the  decoupled  estimators,  theoretical  results  in  [13]  can  be  used  to  prove 
that  if  b  is  estimated  by  minimizing  the  criterion  in  (30)  and  if  that  estimate  is  utilized  for  computing  a  using 
(21),  then  the  resulting  estimates  are  the  unique  and  global  minimizers  of  the  criterion  in  (20).  The  advantage 
of  estimating  the  linear  and  nonlinear  parameters  independently  is  reduction  in  computational  load  because  the 
iterative  part  is  only  with  respect  to  the  p  coefficients  in  b.  Based  on  the  optimal  b,  estimation  of  the  optimal  a 
is  a  simple  linear  least  squares  problem.  But  more  importantly,  a  needs  to  be  computed  only  once. 

IV. 2  :  Algorithm 

The  nonlinear  optimization  criterion  in  (30)  possesses  a  very  useful  matrix  structure.  Specifically,  the  expression 
appears  to  be  a  weighted  quadratic  criterion  in  the  unknown  vector  b.  The  matrices  Z  and  Wj  are  known.  But 
the  weight  matrix  (BTW/Wj B)-1  itself  is  dependent  on  the  unknowns  in  B.  The  computational  algorithm  will 
utilize  this  weighted  quadratic  structure  of  the  criterion  to  formulate  the  iterations.  Specifically,  the  algorithm 
minimizes  the  following  quadratic  error  criterion  at  Ar-th  iteration  step  : 

min  bT[ZT(BT(<:_1)W/Wj’B(fc-1))-1Z]b  A  min  bTRib  (31a) 

b  =  b 

where,  is  formed  by  using  the  estimate  of  b  obtained  at  the  previous  iteration  and 

Ri  A  [ZT(BT^_1^W/WjB(fc_1))_1Z]  is  the  weight-matrix.  An  initial  estimate  of  b  is  necessary  to  start 
the  iterative  process,  b^0^  A  [1  0  ■  •  -0]T  can  be  used  or  the  initial  estimates  could  also  be  found  setting  the 
middle  matrix  (BTW/Wf  B)-1  to  identity,  i.e .,  by  optimizing, 

min  bTZTZb  A  min  bTR2b  (316) 

b  =  b 

where,  the  weight-matrix  R2  A  ZTZ.  In  order  to  ensure  non-trivial  solutions,  the  first  term  of  the  denominator, 
6(0)  is  set  to  unity.  The  computational  algorithm  is  similar  in  nature  to  the  time-domain  counterparts  developed 
recently  [8,  16,  17].  As  outlined  in  the  Appendix,  the  algorithm  has  two  phases.  In  Phase-1,  the  criterion  in  (31a) 
is  minimized  by  neglecting  the  variation  w.r.t.  the  weight  matrix.  Simulation  experience  shows  that  this  Phase 
alone  brings  the  error  quite  close  to  the  minimum.  But  if  necessary,  the  variation  of  the  weight  matrix  may  also 
be  included  by  invoking  Phase-2,  where  the  gradient  of  the  entire  criterion  is  set  to  zero.  Once  the  iterations 
converge,  the  estimated  b  is  used  in  (21)  to  linearly  estimate  the  numerator  coefficient  vector  a. 
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V  :  Simulation  Results 


Two  examples  are  included  to  demonstrate  the  effectiveness  of  the  proposed  algorithm.  The  first  example  considers 
a  Lowpass  filter  design  problem  whereas  the  second  one  designs  a  Notch  filter.  In  all  plots  the  frequency  response 
values  are  displayed  up  to  half  the  sampling  frequency.  For  the  proposed  method,  only  the  Phase- 1  results  are 
given. 

Simulation-1  :  Lowpass  Filter  Design 

Magnitude  response  values  at  56  frequency  points  around  the  unit  circle  were  taken  for  the  matching  purpose.  In 
Fig.  1  the  estimated  response  with  p  =  6  and  q  =  5  for  the  proposed  method  are  shown  by  the  dashed  curve  and 
the  solid  line  represents  the  desired  response.  The  algorithm  converged  in  6  iterations.  For  the  sake  of  comparing 
with  a  widely  used  direct  method,  the  Modified  Yule- Walker  method  [30,  31]  available  in  the  MATLAB  software 
package  was  used  to  design  a  6th  order  filter.  The  magnitude  response  fit  for  this  case  is  shown  as  the  dot-dash 
line  in  Fig.  1. 

Simulation-2  :  Notch  Filter 

A  Notch  Filter  design  problem  was  considered  in  this  case.  The  magnitude  response  values  at  101  frequency 
points  around  the  unit  circle  were  taken.  The  estimated  response  with  10th  order  denominator  and  9th  order 
numerator  as  produced  by  the  proposed  method  as  well  as  the  desired  response  are  shown  in  dB  scale  in  Fig.  2 
in  dashed  and  solid  lines,  respectively.  The  algorithm  converged  in  11  iterations.  The  dash-dot  line  again  shows 
the  fit  when  the  Modified  Yule- Walker  method  [30,  31]  was  used  to  design  the  10th  order  filter. 

Discussion 

The  first  example  has  been  adopted  from  [14,  28].  The  results  presented  above  for  the  proposed  method  did  not 
have  to  make  use  of  any  generic  nonlinear  optimization  technique,  such  as  Newton- Raphson  to  reach  the  final 
optimum.  Also,  during  the  minimization  process,  all  the  coefficients  were  enforced  to  be  real  and  hence  the  filter  is 
readily  realizable.  It  may  also  be  stated  here  that  the  final  designs  were  stabilized  using  the  macro  called  Polystab 
available  in  MATLAB  [29,  30],  where  the  unstable  roots  are  flipped  inside  the  unit  circle.  The  simulations  clearly 
demonstrate  that  the  proposed  method  can  closely  match  arbitrarily  shaped  frequency  response  data  and  it  also 
appears  to  perform  better  than  a  widely  used  method  for  direct  design. 
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APPENDIX  :  Computational  Algorithm 


The  criterion  in  (31a)  is  non-linear  in  b  and  hence  it  can  not  be  minimized  directly.  But  instead  of  using  any 
generic  non-linear  optimization  techniques,  the  inherent  mathematical  structure  of  the  criterion  will  be  utilized  to 
develop  an  iterative  computational  algorithm.  The  algorithm  consists  of  two  phases.  In  Phase-1,  the  variations 
in  the  middle  matrix  (BTW/WjB)  in  (31a)  is  not  taken  into  account  in  the  derivative  calculations,  whereas  in 
Phase-2  the  gradient  of  the  error  norm  in  (31a)  is  set  to  zero. 

Phase- 1 

The  final  form  of  the  error  vector  in  (31a)  is  rewritten  as, 


e6  =  WjB(BT 
A  VZb 

W/WfB)_1Zb 

(Al) 

(A2) 

A  V 

g  :  G 

b 

(A3) 

=  Vg  +  VGb, 

(A4) 

where, 

< 

II  > 

3 

Cd 

ta 

rW/WjB)_1, 

(A5) 

and 

b  A  [6(1)  6(2)  ...  6(p)f. 

(A6) 

If  the  matrix  V  is  treated  as  independent  of  b,  an  expression  for  b  can  be  easily  obtained  by  minimizing  ||e&||2 
w.r.i .  b  as  follows  : 


b  =  —  (VG)#Vg 

=  -  (GtVtVG)'1  GTVTVg.  (A 7) 

But  since  V  does  depend  on  the  elements  in  b,  (A. 7)  can  only  be  computed  iteratively.  At  the  (z  -f  l)-th  step 
of  iteration,  VW  is  formed  using  the  estimate  of  b  found  in  the  z’-th  iteration  step.  This  leads  to  the  following 
iterative  algorithm  for  computing  bt+1  : 


b^1) 


1 

— [F^Gj-^F^g 


(A.8) 


where, 

F(*>  A  GtVt(,)V(<)  (A9) 

The  iterations  are  continued  until  ||bj+i  —  b;||2  <  <5,  where  6  is  an  arbitrarily  small  number.  It  must  be  noted 
here  that  the  iterations  in  (A. 8)  may  not  always  converge  to  the  absolute  minimum  of  the  error  criterion  in  (31a) 
and  hence  the  estimated  b  may  not  be  the  optimum  one.  This  is  because  in  (A. 8)  the  variability  of  V  w.r.i.  b 
had  been  ignored  while  minimizing  ||e||2.  To  achieve  the  optimum,  the  gradient  of  the  complete  expression  of 
||e||2  must  be  set  to  zero.  If  desired,  this  can  be  done  in  Phase-2  of  the  algorithm  which  is  outlined  next.  It  may 
be  noted  here  that  the  simulation  studies  indicate  that  the  Phase-1  of  iterations  using  (A. 8)  perform  an  excellent 
job  of  bringing  the  estimate  very  close  to  the  optimum.  Once  the  estimates  of  b  converge,  a  is  computed  using 
(21). 
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Phase-2 


In  this  phase,  the  derivative  of  the  matrix  V  w.r.t.  b  is  taken  into  consideration  while  minimizing  the  fitting 
error  norm.  By  setting  the  derivative  of  the  squared  norm  in  (31a)  to  zero,  we  obtain  the  updated  at  the 

(i  +  l)-th  iteration  as, 

b(<+1)  =  -  [S^G]-1[S^]g  (A. 10) 

where  (suppressing  the  superscript  W), 


S  A  LtV  +  GtVtV, 

r  dv  i  |  dv  „ 


(,4.11a) 

(A.116) 


dv  d 

db(k)  =  db(, 


=  ^[W'B(BTW/W/TB)_1]  =  W^^)(BTW/W^B)_1  "  W'B 

L 


W/Wf  B 


+  btw/wJ’  — —  (btw/wJb)-1 


(A. lie) 


and  has  the  same  form  as  the  B  matrix  defined  in  (10)  but  it  is  filled  with  all  zeros  except  at  the  locations 
where  b(k)  appears.  For  example, 


■0  • 

•  1 

0  • 
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0  • 

•  0 

1  • 

•  0 

0  ' 

•  0 

0  ■ 

.  1 

0  • 

•  0 

0  ' 

.  0 

•  0 

.0  • 

•  0 

0  • 

•  o. 

1  €IR 


NxN-q- 1 


(A.12) 


Once  b(’+1)  is  found  using  (A. 10),  b(,+1)  can  be  formed  as, 

b^  = 


b^1) 


-[SWGj-^SWjg 


(A. 13a) 
(A. 136) 


This  minimization  phase  continues  until  b,+1  ~  b*  is  reached  and  this  optimum  b  vector  corresponds  to  a 
minimum  of  the  error  surface  of  ||ej||2. 
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Section  -  3.3  :  OPTIMAL  ESTIMATION  OF  THE  PARAMETERS  OF  ALL-POLE  TRANSFER 

Functions 

Summary 

An  algorithm  is  proposed  for  optimal  estimation  of  the  parameters  of  Auto-Regressive  (AR)  or  all-pole 
transfer  function  models  from  prescribed  impulse  response  data.  The  transfer  function  coefficients  are  estimated 
by  minimizing  the  ^-norm  of  the  exact  model  fitting  error.  Existing  methods  either  minimize  equation  errors  or 
modify  the  true  non-linear  fitting  error  criterion.  In  the  proposed  method,  the  multidimensional  nonlinear  error 
criterion  has  been  decoupled  into  a  purely  linear  and  a  nonlinear  subproblem.  Global  optimality  properties  of  the 
decoupled  estimators  have  been  established.  For  data  corrupted  with  Gaussianly  distributed  noise,  the  proposed 
method  produces  Maximum-Likelihood  Estimates  (MLE)  of  the  AR- parameters.  The  inherent  mathematical 
structure  in  the  non-linear  subproblem  is  exploited  in  formulating  an  efficient  iterative  computational  algorithm  for 
its  minimization.  The  proposed  algorithm  provides  an  useful  computational  tool  based  on  appropriate  theoretical 
foundation  for  accurate  modeling  of  all-pole  systems  from  prescribed  impulse  response  data.  The  effectiveness  of 
the  algorithm  has  been  demonstrated  with  several  simulation  examples. 

l.  Introduction 

Parameter  estimation  of  unknown  discrete-time  linear  systems  is  a  fundamental  problem  in  digital  signal 
processing.  Parametric  models  overcome  the  infinite  dimensionality  problem  of  non-parametric  models  with 
parsimonious  representation  of  systems  in  terms  of  only  a  finite  number  of  parameters.  Over  the  last  few  decades 
these  problems  have  been  addressed  in  a  large  body  of  work  in  many  different  fields  [1-17,  22,  24-38,  40-47]. 
Among  many  parametric  models  used  in  signal  processing,  Auto- Regressive  (AR)  or  all-pole  model  is  one  of  the 
most  effective  and  practical  representations. 

The  AR-parameter  identification  problem  arises  both  in  stochastic  and  deterministic  time-series  analysis. 
There  are  probably  two  primary  reasons  for  the  wide  popularity  of  AR  modeling  in  statistical  time  series  analysis. 
Firstly,  according  to  Kolmogorov  Theorem,  any  minimum  phase  transfer  function  H (z)  can  be  represented  by  a 
possibly  infinite  order,  stable  minimum  phase  AR-model  [9,  10].  This  important  theorem  implies  that  even  if  an 
AR  model  is  picked  erroneously,  the  unknown  Power  Spectral  Density  can  still  be  matched  closely  as  long  as  a 
‘large  enough’  AR  model  order  is  chosen.  But  the  second  and  the  main  reason  for  the  popularity  of  AR-modeling 
is  that  it  is  possible  to  obtain  reasonably  good  suboptimal  estimates  of  the  unknown  AR-parameters  by  solving  a 
simultaneous  set  of  linear  equations. 

Modeling  human  vocal-tracts  as  all-pole  systems  and  the  corresponding  Speech  signal  as  AR-process  is  one 
of  the  most  important  applications  of  AR-modeling  [4,  8].  Furthermore,  two  important  modeling  philosophies, 
viz.,  Linear  Prediction  (LP)  and  Maximum  Entropy  methods  essentially  produce  the  AR-parameters  as  their 
estimates,  regardless  of  the  true  underlying  signal  model. 

This  Section  deals  with  the  problem  of  estimating  the  parameters  of  an  all-pole  transfer  function  to  match 
a  prescribed  or  desired  impulse  response  specification.  The  least-squares  Impulse  Response  (IR)  model  fitting 
error  has  been  chosen  as  the  objective  optimality  criterion.  Many  well-known  techniques  developed  for  statistical 
time-series  analysis  have  been  used  successfully  in  the  deterministic  case  also  [7,  10].  AR-model  fitting  may 
be  considered  a  special  case  of  estimating  the  unknown  parameters  of  general  ARMA  (or  pole-zero)  models. 
ARMA  parameter  estimation  is  known  to  be  a  multidimensional  nonlinear  optimization  problem  and  there  have 
been  extensive  work  on  this  subject  [1-7,  10,  12,  13,  15,  16,  24-31,  34-37,  41-43,  47].  In  one  of  the  earlier 
works,  Kalman  [1]  had  proposed  a  linearized  and  approximate  ‘equation  error’  minimization  technique  which 
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produces  suboptimal  estimates.  Several  two-step  procedures  where  the  denominator  and  numerator  polynomials 
are  estimated  separately,  have  also  been  proposed  [2,  30].  In  these  methods,  the  denominator  is  first  estimated 
by  minimizing  an  ‘equation  error’  and  then  the  numerator  is  found  by  minimizing  a  linearized  ‘fitting  error’  [2] 
or  by  setting  the  leading  error  samples  to  zeros  [30].  A  thorough  coverage  on  filter  design  by  modeling  may  be 
found  in  [7]. 

Equation  error  minimization  is  a  commonly  used  optimization  procedure  for  estimating  AR-parameters. 
In  fact,  the  well-known  linear  prediction  (LP)  coefficients  [7-10]  are  estimated  by  minimizing  equation  errors. 
Linear  predictors  have  extensive  usefulness  in  speech  analysis,  synthesis  and  coding.  Many  practical  and  efficient 
algorithms  are  available  for  LP  parameter  estimation.  Among  these,  the  ‘Autocorrelation  Method’  (AM)  and 
the  ‘Covariance  Method’  (CM)  are  most  popular.  CM  and  AM  do  not  produce  optimal  estimates  in  the  sense 
that  the  model  fitting  error  norm  is  not  minimized  in  either  case.  In  contrast,  Steiglitz  and  McBride  (SMM)  had 
proposed  a  modified  fitting  error  minimization  criterion  for  estimating  the  coefficients  of  general  ARMA  models 
[3,  4,  7].  SMM  has  also  been  adapted  for  AR  parameter  estimation  [7].  In  absence  of  any  exact  model  fitting  error 
criterion,  SMM  ha s  established  itself  as  the  standard  method  for  AR  and  ARMA  parameter  estimation  problems 
[3,  4,  7,  10,  12,  22,  25,  34,  40,  47].  In  [5],  a  decoupled  exact  fitting  error  minimization  approach  has  also  been 
proposed  by  Evans  and  Fischl  (EFM).  But  their  algorithm  is  applicable  only  in  the  case  of  strictly  proper  ARMA 
models  where  the  number  of  poles  must  be  exactly  one  more  than  the  number  of  zeros.  Consequently,  the  optimal 
EFM  can  be  applicable  for  identifying  first-order  AR-models  only.  The  proposed  optimal  algorithm  has  no  such 
restrictions. 

The  proposed  algorithm  originates  from  a  recently  developed  optimal  method  (OM)  for  general  ARMA 
modeling  [6].  Unlike  EFM,  the  decoupled  fitting  error  minimization  approach  in  [6]  is  applicable  for  ARMA 
models  with  arbitrary  numbers  of  poles  and  zeros.  Furthermore,  in  contrast  to  the  methods  in  [1-4,  24,  30],  no 
linearization  or  modification  of  error  criterion  is  introduced  in  the  theoretical  derivation  of  the  least-squares  model 
fitting  criterion.  In  this  Section,  the  complete  derivation  of  the  optimal  solution  for  the  AR  case  (OM-AR)  is  being 
presented  for  the  first  time.  It  is  also  shown  that  if  the  observation  data  is  composed  of  true  impulse  response 
corrupted  by  Gaussianly  distributed  noise,  then  the  proposed  optimization  produces  the  Maximum- Likelihood 
estimates  (MLE)  of  the  AR  parameters.  For  other  types  of  noise  or  deviations  least-squares  estimates  (LSE)  are 
found. 

A  critical  step  in  the  theoretical  derivation  of  the  error  criterion  is  to  decouple  the  multidimensional  criterion 
into  a  non-linear  problem  for  the  AR-parameters  and  a  linear  problem  for  the  numerator  coefficient.  In  the 
decoupled  form,  the  fitting  error  is  found  to  be  related  to  an  equation  error  which  is  different  than  the  ones  that 
appear  in  CM  or  AM.  But  the  form  of  the  equation  error  is  shown  to  be  mathematically  appropriate  for  the  AR 
case.  The  non-linear  criterion  possesses  inherent  matrix  prefiltering  structure  which  directly  leads  to  formulating 
an  efficient  iterative  computational  algorithm  for  its  minimization.  Several  simulation  examples  demonstrate  the 
superior  performance  of  the  proposed  approach  when  compared  to  some  of  the  existing  suboptimal  methods. 

The  Section  is  arranged  as  follows  :  in  Subsection  II,  the  problem  is  defined,  the  connection  with  MLE  is 
established  and  some  existing  results  are  briefly  outlined.  In  Subsection  III  the  error  criterion  is  theoretically 
derived  for  the  AR  case  and  the  computational  algorithm  is  presented.  In  Subsection  IV,  several  simulation 
examples  are  given.  Finally,  in  Subsection  V,  some  concluding  remarks  are  given. 

II.  Problem  Statement  and  Previous  Results 

The  2-domain  transfer  function  for  an  auto-regressive  model  can  be  represented  as, 


where  the  coefficient  of  the  z°  term  in  the  denominator  has  been  assumed  to  be  unity  without  any  loss  of  generality. 
As  an  example,  H(z)  may  represent  the  transfer  function  of  human  vocal  tract  which  is  commonly  modeled  as 
an  all-pole  model.  The  model  order  p  is  assumed  to  be  known.  In  case  of  speech  signals,  for  example,  a  lot 
of  experience  and  knowledge  is  already  available  and  the  value  of  p  =  10  or  8  is  usually  chosen.  An  equivalent 
representation  of  the  transfer  function  H(z)  can  also  be  written  in  terms  of  its  impulse  response  as, 

H(z)  =  h(0)  +  h{l)z-x  4 -  ■ . .  +  fe(JV  -  2 +  h(N  -  1  )z^N~^  +  •  •  • .  (2) 

The  first  N  significant  samples  of  H(z)  can  be  stacked  in  a  vector  form  as, 

h  A  [h( 0)  h(l)  ■■■  h(N-  l)f .  (3) 


Next,  the  vector  containing  the  N  samples  of  the  ‘prescribed’  or  ‘desired’  impulse  response  data  is  denoted  as, 

hP  A  [MO)  Ml)  •••  hP(N-l)]T.  (4) 

The  desired  IR  data  vector  may  represent  the  impulse  response  of  vocal  tract.  With  these  definitions,  the  problem 
addressed  in  this  Section  may  be  stated  as  follows  : 

Given  a  desired  impulse  response  hp,  the  goal  is  to  obtain  the  optimal  estimates  of  the  model  parameters 
no  and  d  by  minimizing  the  following  least-squares  IR  model-fitting  criterion  : 


mm  ||e 

n0,d 


e|'2^  • 
r/.N.  /  1,  i  =  0 

<W=  to,  i#0, 

e  A  hp  —  h  and 


where, 


d  A  [1  di 


(5) 

(5a) 

(56) 

(5c) 


The  notation,  denotes  the  response  of  the  system,  when  driven  by  an  input  sequence,  <5(i). 

Clearly,  the  criterion  in  (5)  attempts  to  minimize  the  squared  error  between  the  desired  and  the  estimated  IR  and 
hence,  it  can  be  expected  to  produce  more  accurate  model  than  some  well-known  AR  modeling  methods  (outlined 
below)  which  only  minimize  ‘equation  errors’.  The  least-squares  problem  in  (5)  is  known  to  be  nonlinear  in  d  and 
standard  nonlinear  optimization  algorithms  have  been  utilized  before  in  [15,  25-29,  36].  It  should  be  emphasized 
that  if  the  given  IR- vector  hp  is  composed  of  the  true  IR- vector  h  and  additive  Gaussianly  distributed  noise 
or  deviations  then  the  minimization  criterion  in  (5)  is  exactly  equivalent  to  the  maximization  of  the  Likelihood 
criterion  [see  ref.  10,  pp.  242-248].  Hence,  for  such  a  scenario  the  algorithm  proposed  in  this  Section  produces  the 
MLE  of  the  AR-parameters.  For  all  other  types  of  noise  and  deviations  the  Least-Squares  Estimates  are  found. 
It  may  also  be  noted  that  the  MLE  result  in  [10]  is  primarily  based  on  the  works  in  [5,  13,  44]  where  only  the 
strictly  proper  ARMA  case  was  considered.  The  MLE  for  transfer  functions  with  arbitrary  number  of  poles  and 
zeros  has  been  presented  recently  in  [6]. 

In  many  applications,  such  as  in  linear  prediction  of  speech  signals  [4,  8],  only  the  estimation  of  the  AR- 
parameters  is  of  primary  concern.  The  two  most  commonly  used  LP  algorithms,  AM  and  CM,  do  not  solve  the 
ideal  problem  stated  in  (5)  whereas  SMM  attempts  to  solve  the  ideal  problem  by  appropriate  modification  of  the 
criterion  in  (5)..  These  three  approaches  are  briefly  summarized  next. 


102 


Covariance  Method  [CM] 


-1 

- 1  - 

di 

J 

.dp  _ 

=  e 


The  ^2-norm  of  the  following  equation  error  is  minimized  [7-10]  : 

hP(p)  hP(p- 1)  •••  hP(  0) 

_hp(N  —  1)  hp(N  —  2)  •••  hP(N-p-r 

or,  Hcjv/d  A 

Note  that  H cm  is  filled  with  available  IR  data  only  and  hence  may  be  considered  an  ‘exact’  equation  error. 

Auto-correlation  Method  [AM] 

In  this  case,  the  ^-norm  of  the  following  equation  error  is  minimized  [7-10]  : 
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or, 


A  eeq 


AM 


(7b) 


The  zeros  in  the  upper  and  lower  triangles  of  H am  are  not  part  of  the  prescribed  IR-vector  hp  and  hence  efqM 
is  not  an  exact  equation  error. 

It  can  be  observed  from  (6)  and  (7)  that  the  equation  error  for  CM  uses  windowed  data  without  making 
any  prior  assumptions  about  the  data  outside  the  observed  window  {hp(0)  ...  hp(N  —  1)}.  On  the  other  hand, 
AM  uses  unwindowed  data  but  sets  the  data  outside  the  observation  frame  to  zero.  Because  of  this  reason,  AM 
usually  produces  less  accurate  estimates  than  CM.  But  it  should  also  be  noted  that  even  though  e^M  is  not  an 
exact  equation  error,  one  of  the  significant  advantages  of  using  AM  is  that  the  computationally  efficient  Levinson- 
Recursion  algorithm  can  be  utilized.  In  case  of  CM,  a  somewhat  less  efficient  algorithm,  Cholesky  decomposition 
can  be  used  [7-10].  Furthermore,  the  AR  coefficients  obtained  by  minimizing  the  norm  of  e^M  produce  a  stable 
transfer  function. 

Steiglitz-McBride  Method  [SMM] 

This  method  was  originally  developed  for  general  ARMA  parameter  identification  but  it  has  also  been  adapted 
for  AR  parameter  identification.  For  the  AR  case,  the  following  modified  fitting  error  criterion  is  optimized  [7], 


(8) 


The  estimate  D(z)  obtained  at  any  iteration  step  is  used  as  a  prefilter  for  obtaining  the  updated  estimates  at  the 
succeeding  iteration.  Equation  (8)  closely  approximates  the  criterion  in  (5)  and  both  are  identical  if  D(z)  =  D(z). 
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But  the  advantage  of  using  (8)  is  that  the  unknown  parameters  in  d  and  no  can  be  estimated  by  solving  a  set  of 
simultaneous  linear  equations.  It  may  be  noted  here  that  in  [7],  the  numerator  coefficient  no  had  been  assumed 
to  be  unity  but,  in  general,  that  may  not  be  the  case.  The  derivation  of  the  proposed  fitting  error  optimization 
scheme  is  in  order. 

III.  Problem  Formulation  and  Algorithm  Development 

In  this  Subsection,  the  multidimensional  optimization  problem  in  (5)  is  decoupled  into  a  linear  estimation 
problem  for  n0  and  a  non-linear  optimization  problem  for  d.  Let  Hd(z)  be  the  inverse  filter  corresponding  to 
D(z),  i.e., 

D(z)Hd{z)  =  1.  (9) 

In  time  domain,  this  corresponds  to  a  convolution  operation  where  the  are  finite  and  the  hd(ny  s  are  infinite 
in  extent.  The  first  N  significant  terms  of  this  convolution  operation  may  be  expressed  in  matrix  notation  as, 


DHd  : 

=  I N 

(10) 

where,  Ijy  denotes  an  N  x  N  identity  matrix, 
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hd(N-2)  hd(  0)J 

Using  (9),  the  expression  in  (1)  can  be  rewritten  as, 

m  =  noHd(z).  (12) 

Equating  the  first  N  coefficients  of  equal  powers  of  z~1  in  both  sides  of  (12)  and  using  vector  notation, 


h  A  nohd, 

where,  is  also  the  first  column  of  defined  in  (lib),  Le., 

(13) 

hd  A  [MO)  Ml)  ••  hd(N  -l)f. 

With  these  definitions,  the  problem  stated  in  (5)  can  be  rephrased  as, 

(14) 

min||e||2  A  min||hp  —  noh^l]2, 

no,d  =  Uo,<I 

where,  the  error  vector  is  defined  as, 

(15) 

e  A  hp  —  nohrf. 

(16) 
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It  is  clear  from  (16)  that  the  error  e  is  linearly  related  to  n0  whereas  e  is  non-linear ly  related  to  d  through  the 
vector  h<*.  In  this  form,  it  is  apparent  that  the  present  problem  belongs  to  a  class  of  mixed  optimization  problems 
where  the  linear  and  nonlinear  variables  appear  separately.  This  class  of  problems  has  been  studied  extensively 
by  numerical  analysts  [18-21].  In  their  work,  the  main  objective  had  been  to  optimize  the  two  sets  of  variables 
independently.  Their  argument  goes  as  follows.  If  hj  (z.e.,  d)  is  known,  then  no  can  be  optimally  estimated  by 
the  minimization  of  the  criterion  in  (15)  and  the  resulting  least-squares  estimate  will  be  given  by, 

n0  A  hfhp,  (17) 

where  #  denotes  pseudo-inverse  operation  defined  as,  hjf  A  (hjh(i)"'1hj.  In  practice,  d  will  not  be  known  and 
it  has  to  be  estimated.  Plugging  ho  back  in  (15),  the  optimization  criterion  for  d  can  be  found  as, 

min ||hp  -  n0h,j||2  =  min||hp  -  (h,jhf)hp||2  (18a) 

rca,d  d 

=  min||(Iftr  -  Phd)hp||2,  (186) 

a 

where  Phd  denotes  projection  matrix  defined  as,  Phd  A  ha(hjh^)  xhJ.  For  a  larger  class  of  multidimensional 
nonlinear  optimization  problems,  it  has  been  proved  in  Theorem  2.1  of  [18]  that  if  d  is  estimated  by  minimizing 
the  criterion  in  (18)  and  if  that  estimate  is  utilized  for  computing  no  using  (17),  then  the  resulting  estimates 
are  the  unique  and  global  minimizers  of  the  criterion  in  (15).  Hence,  the  original  optimization  problem  in  (5) 
is  identical  to  the  decoupled  estimators  in  (17)  and  (18).  This  type  of  decoupled  optimization  of  linear  and 
non-linear  subproblems  had  been  utilized  before  in  [5,  13,  45,  46]  for  strictly  proper  ARM  A  case  and  in  [6]  for 
the  general  ARMA  case.  The  derivation  for  the  AR  case,  as  given  here,  appears  to  be  new. 

The  AR-parameters  in  d  are  related  to  the  error  criterion  in  a  complicated  manner  through  Phd.  Hence,  the 
direct  optimization  of  (18)  w.r.t .  d  would  require  taking  resort  to  standard  non-linear  optimization  techniques 
such  as  Newton-Raphson  or  Gauss- Newton  methods.  Instead,  following  the  strategy  used  in  [6]  for  the  general 
ARMA  case,  the  criterion  in  (18)  is  reparameterized  by  relating  it  directly  to  the  coefficients  in  d.  Appropriate 
partitioning  of  the  matrices  D  and  gives, 

1  0  .  0  0 

dx  1  .  0  0 

dp  dp  _  i  *  •  •  1  ■  •  •  0 

0  •  • .  dp  .  1 

Mo)  I  o  •  •  •  o  I 
Mi)  I  Mo)  o 

Hd  A  :  |  :  :  A  hd  :  H'd  .  (196) 

hd(N  -  1)  |  hd(N  —  2)  MO)  ~~ 

Using  these  notations,  the  expression  in  (10)  can  be  rewritten  as, 

hd ;  H'd  =  i*, 

1  I  0lx(jv-i) 

°(AT-l)xl  I  I(JV-l) 


■  dlhd  I  d^H'  ■ 
_  BThd  I  BTH'd  _ 


(19c) 

(19d) 


r  dT  1 

A -  and  (19a) 

=  Bt 
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The  bottom-left  corner  element  shows  that  the  N  x  (N  -  1)  matrix  B  and  the  vector  hd  are  orthogonal,  i.e ., 
BTh d  —  0.  Also  by  construction, 

rank(  B)  +  rank(hd)  =  N.  (20) 

Hence,  using  a  theorem  on  projection  matrices  [39], 

Pb  +  Phd  =  I  n-  (21) 

Using  this  relationship  in  (18b),  the  following  equivalent  forms  of  reparameterized  optimization  criterion  are 
obtained, 

min||(lAr  -  Phjhp||2  =  min  ||PBhj>||2, 

a  d 

=  min||B(BTB)-1BThi>||2, 

d 

=  min  ||B(BTB)-1ee?||2, 

d 

=  mineJ?(BTB)-1ee?, 

where,  eeq  is  an  equation  error  defined  a s, 

eeq  A  BThp.  (23) 

This  equation  error  can  also  be  rewritten  as, 

hp(  1)  hp{ 0)  *  .•  0  "I  pX  “ 

rp  *  )  *  *  (l\ 

eeq  =  B  hp  =  •  •  **  :  .  (24a) 

hP(N-l)  hP(N  —  2)  ...  hP(N-p-  1)  : 

-  dp  _ 

A  H^pd.  (246) 

A  few  observations  may  be  made  here  regarding  the  equation  error  defined  in  (23).  Clearly,  eeq  differs  from  the 
equation  errors  used  in  CM  and  AM  as  defined  in  (6)  and  (7),  respectively.  The  equation  errors  in  those  cases 
were  formed  in  somewhat  ad  hoc  manner  on  the  basis  of  two  types  of  autocorrelation  estimates  [7,  9,  10].  On 
the  other  hand,  the  particular  form  of  equation  error  in  (24a)  resulted  from  purely  mathematical  consequences 
of  the  AR  case  under  consideration.  In  particular,  if  the  prescribed  response  hp  happens  to  be  an  exact  impulse 
response  of  a  p-th  order  AR  transfer  function,  then  the  equation  error  in  (24a)  will  be  identically  equal  to  zero, 
but  the  same  will  not  be  true  for  efqM  in  (7).  The  equation  error  for  CM  appearing  in  (6)  will  also  be  zero  but 
ignores  the  information  contained  in  the  upper  ( p  —  1)  equations  of  (24a).  From  this  discussion,  it  can  be 
concluded  that  more  accurate  estimates  may  be  obtained  if  the  equation  error  in  (23)  is  used  for  the  AR  case. 
Minimization  of  this  equation  error  will  be  utilized  later  in  the  computational  algorithm  for  obtaining  the  initial 
estimate  of  d. 

Using  (24b)  in  (22d),  the  reparameterized  criterion  can  be  expressed  in  the  following  useful  form, 

mindTH5R(BTB)-1H^Hd.  (25) 

According  to  Theorem  2.1  of  [18],  the  denominator  vector  d  causing  the  minimum  of  the  criterion  in  (25)  is  the 
desired  optimum  d°.  The  minimized  error  can  then  be  found  from, 

e°  =  PB°hp,  (26) 


(22a) 
(226) 
(22c) 
(22  d) 
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where,  B°  is  constructed  by  using  the  optimum  d°.  The  optimum  estimate  of  the  impulse  response  is  then, 


h°  =  hp  -  e°.  (27) 

From  [18],  it  can  be  also  be  inferred  that  if  is  formed  using  d°  then  the  optimal  rig  can  be  obtained  using  (17) 
as, 

ng  A  hjfh P.  (28a) 

Next  it  is  shown  that  instead  of  using  (28a),  the  optimal  numerator  coefficient  can  be  found  in  a  more  straight¬ 
forward  manner  as, 

»5  =  h°(  0),  (286) 

where  h°( 0)  is  the  first  sample  of  h°,  the  optimal  impulse  response  estimate  found  in  (27).  In  order  to  demonstrate 
this  equivalence,  equation  (28b)  is  rewritten  as, 


ng  =  hfd°,  (29a) 

where, 

Note  that  the  first  term  in 
rewritten  as, 


dlh°, 

(30a) 

d„(hp  -  e°), 

using  (27), 

(306) 

dl(hP  -hP  +  h°d 

hjf  hp),  using  (18a)  and  (26) 

(30c) 

(djh^hfhp, 

(30d) 

h2#hp, 

(30e) 

where,  the  last  equality  uses  the  fact  that,  djh 0 d  =  1,  which  appears  in  the  upper-left  partition  of  (19c).  This 
completes  the  proof  of  equivalence  between  the  expressions  in  (28a)  and  (28b).  It  should  be  noted  that  (29)  may 
be  preferable  over  (28a)  for  computing  no  because  the  computation  of  and  the  pseudo-inverse  solution  required 
in  (28a)  can  be  avoided,  whereas  calculation  of  the  optimal  h°  in  (27)  may  be  a  necessary  step.  Equations  (22)  and 
(29)  are  the  two  desired  decoupled  formulae  for  estimation  of  the  coefficients  of  the  denominator  and  numerator 
polynomials  of  the  AR-model.  It  should  be  mentioned  that  unlike  the  decoupled  forms  of  SMM  given  in  [7]  and 
[34],  no  approximations  were  introduced  in  deriving  the  decoupled  estimators  in  (22)  and  (29).  A  computational 
algorithm  for  minimization  of  the  criterion  in  (22)  is  outlined  next. 

Computational  Algorithm 

The  criterion  in  (22)  is  non-linear  in  d  and  hence  it  can  not  be  minimized  directly.  Standard  gradient-based 
non-linear  optimization  techniques  such  as  Newton- Raphson  or  Gauss-Newton  algorithms  could  be  used.  But 
these  algorithms  utilize  only  the  first  few  terms  of  Taylor  series  and  are  known  to  be  highly  sensitive  to  the  choice 
of  the  initial  estimates.  But  it  can  be  observed  from  (25)  that  the  error  criterion  possesses  a  good  deal  of  matrix 
structure.  Specifically,  the  expression  appears  to  be  a  weighted  quadratic  criterion  in  d,  except  that  the  weight 
matrix  (BTB)~1  itself  is  dependent  on  the  unknowns  in  d.  This  inherent  mathematical  structure  of  the  criterion 
Avill  be  utilized  to  develop  an  iterative  computational  algorithm.  The  algorithm  is  similar  to  the  ones  for  ARMA 
cases  appearing  in  [5,  6,  13].  Here  the  complete  derivation  for  the  AR  case  will  be  given. 


h?  A  [h°{ 0)  0  •  •  •  Of.  (296) 

d  is  always  1.  Using  the  partitioning  notation  in  (19a),  equation  (29a)  can  also  be 
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In  order  to  initiate  the  iterative  algorithm,  d  is  first  estimated  by  minimizing  the  ^-norm  of  the  equation 
error  eeq  defined  in  (24a).  Partitioning  the  equation  error  eeq  can  be  rewritten  as  follows, 


eeq  =  BThp  A 


hP(l) 


MO) 


I 


hP(N- 1)  |  hp(N  —  2)  •••  hp(N-p-l) 


■  1  - 

di 

d 2 

■ 

- 1 

'*  ^ 

_ 1 

g  :  G 


d. 


(31a) 


(316) 


Minimizing  ||e  e?H2  w.r.t.  d,  the  following  initial  estimate  is  obtained 


d(°)  = 


1 

— G#g 


(32) 


This  estimate  will  be  utilized  for  initiating  the  iterative  computational  algorithm.  The  final  form  of  the  error 
vector  in  (22a)  is  rewritten  as, 


where, 


e  =  B(BtB)_1 

BTh  P 

(33a) 

A  WBThP 

using  (24), 

(336) 

=  WH,fid 

using  (31b), 

(33c) 

=  W 

g  :  G 

d 

(33d) 

=  Wg  +  WGd, 

(33e) 

W  A  B(BtB)“ 

1 ,  and, 

(33/) 

d  A  [d 

i  d2 

...  dp]T. 

(33</) 

If  the  matrix  W  is  treated  as  independent  of  d,  minimization  of  ||e||2  w.r.t.  d  results  in  the  following  estimate  : 


d  =  -  (WG)#Wg 

=  -  (GTWTWG)_1GTWTWg. 


(34) 


But  W  does  have  dependence  on  the  elements  in  d  and  hence  the  estimate  in  (34)  can  only  be  computed  iteratively. 
The  estimate  of  d  found  in  the  2-th  iteration  step  is  used  in  (33f)  to  form  which  is  then  utilized  at  the 

(i  +  l)-th  step  of  iteration  to  compute  dt+1  as  follows  : 


d<i+1>  = 


. — [X(*>  G]“ 1  [X^jg 


where, 


xW  A  GtWt(,)W(‘) 

=  gt(bt(,)b(<))-1. 
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(35a) 


(356) 

(35c) 


The  iterations  are  continued  until  ||dj+i  —  d*||2  <  6,  where  S  is  an  arbitrarily  small  number. 

It  must  be  noted  here  that  the  iterations  in  (35)  may  not  always  converge  to  the  absolute  minimum  of 
the  error  criterion  in  (5)  and  hence  the  estimated  d  may  not  be  the  optimum  one.  This  is  because  in  (35)  the 
variability  of  W  w.r.t.  d  has  been  ignored  while  minimizing  ||e||2.  To  achieve  the  optimum,  the  gradient  of  the 
complete  expression  of  ||e||2  must  be  set  to  zero.  If  desired,  this  may  be  done  in  a  second  phase  of  the  algorithm 
which  is  outlined  in  the  Appendix.  Invoking  Phase-2  will  assure  that  at  least  a  local  minimum  will  be  achieved. 
But  it  may  be  noted  here  that  the  simulation  studies  indicate  that  the  Phase- 1  of  iterations  using  (35)  does 
an  excellent  job  of  bringing  the  estimate  very  close  to  the  optimum.  It  will  be  shown  in  Subsection  IV  that 
the  Phase-2,  if  invoked,  causes  almost  insignificant  changes  in  the  d  vector  and  the  minimized  error  norm.  In 
simulations,  the  convergence  was  found  to  be  quite  rapid  in  both  the  phases.  Once  the  estimates  of  d  converge, 
n0  is  found  using  (26),  (27)  and  (29),  in  sequence. 

Discussion 

The  major  computational  burden  of  the  algorithm  is  in  performing  the  iterative  refinement  in  (35),  where, 
at  each  iteration  step  an  (N  ~  1)  x  (N  —  1)  matrix  (BTB)  needs  to  be  inverted.  But  (BTB)  is  a  banded  and 
symmetric  matrix  which  can  be  inverted  using  computationally  efficient  Cholesky  decomposition  [8,  10].  Further 
reduction  in  computation  is  also  possible  because  though  (BTB)  is  not  purely  Toeplitz,  a  major  (N  —  p)  x  (N  —  p) 
diagonal  block  is  symmetric-banded-Toeplitz  and  this  block  can  be  inverted  with  0[(N  —  p )  log (N  —  p )]  T  0\p 2] 
operations  [23].  The  other  (p  —  1)  x  (p  —  1)  diagonal  block  is  symmetric  and  can  be  inverted  with  0[(p  —  l)2] 
operations.  Furthermore,  the  non-diagonal  blocks  contain  mostly  zero  elements.  Hence,  using  the  block  matrix 
inversion  formula  due  to  Schur  [48],  this  matrix  inversion  can  be  computed  with  less  than  0[(N  —  l)2]  operations. 
It  may  also  be  noted  that  in  case  of  SMM  the  calculation  of  IR  of  the  inverse  filter  and  data  filtering  are  required 
at  every  step  of  iteration,  whereas  the  proposed  method  uses  the  estimated  d  directly  to  form  the  B  matrix. 

The  LS  error  criterion  defined  in  (5)  attempts  to  match  only  the  first  N  available  samples  of  hp(n).  No  explicit 
assumption  has  been  made  about  the  unobserved  samples,  but  the  estimated  rational  transfer  function  essentially 
extends  the  impulse  response  beyond  the  observations.  It  may  be  noted  here  that  minimum  phase  property  can 
not  be  guaranteed  with  the  AR-parameter  estimates  produced  by  the  proposed  algorithm.  Extensive  simulation 
studies  indicate  though  that  with  converging  IR  sequences,  the  algorithms  always  produced  stable  solutions.  It 
should  also  be  pointed  out  that  among  existing  methods,  only  the  autocorrelation  method  can  guarantee  stable 
solutions.  But  AM  uses  windowed  data  and  the  IR  fit  with  the  estimates  is  usually  not  very  accurate  because  the 
original  least-squares  IR  error  criterion  is  not  minimized.  To  ensure  minimum  phase  solution,  AM  can  be  used 
(instead  of  (32))  to  obtain  the  initial  estimates  for  starting  the  iterative  AR-algorithm.  If  the  estimates  obtained 
from  the  iterative  scheme  becomes  maximum  phase  at  any  iteration  step  of  the  AR-algorithm,  the  iterations  can 
be  terminated  at  that  stage.  The  estimate  found  at  the  preceding  iteration  should  be  accepted  as  the  best  possible 
minimum  phase  solution  that  minimizes  the  optimal  LS  criterion  in  (5). 

The  model  order  selection  problem  has  not  been  addressed  in  this  work.  It  appears  that  for  this  essentially 
deterministic  problem,  Akaike  Information  Criterion  (AIC)  or  Minimum  Description  Length  Criterion  (MDL) 
may  not  be  applicable.  But  these  criteria  may  be  utilized  when  the  prescribed  impulse  response  data  consists  of 
true  impulse  response  embedded  in  Gaussianly  distributed  noise. 

The  algorithm  presented  in  this  Section  may  also  be  quite  useful  for  estimating  MA  filter  coefficients. 
Presently,  the  most  effective  algorithm  for  MA  modeling  is  Durbin’s  method  [11]  which,  in  fact,  relies  on  two 
steps  of  AR  parameter  estimation.  Traditionally,  AM  is  utilized  in  both  steps  of  Durbin’s  method  because  it 
produces  minimum- phase  polynomials  [7,  9-11].  But  the  estimates  obtained  using  AM  may  not  be  optimal  be¬ 
cause  the  true  impulse  response  fitting  error  norm  is  not  minimized.  But  the  algorithm  presented  here  produces 
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optimal  least-squares  AR  filter  coefficients  from  prescribed  impulse  response  data.  Hence,  it  can  be  expected  that 
the  introduction  of  the  proposed  AR  algorithm  in  one  or  both  stages  of  Durbin’s  algorithm  may  produce  more 
accurate  MA  parameter  estimates. 

It  has  been  shown  in  [7]  that  the  original  SMM  can  also  be  decoupled  into  a  linear  and  a  non-linear  subprob¬ 
lems.  In  a  recent  paper  [34],  the  strictly  proper  case  of  the  original  SMM  has  been  decoupled  somewhat  differently 
than  in  [7].  But  more  interestingly,  it  has  also  been  demonstrated  that  the  non-linear  part  of  the  decoupled  SM 
criterion  has  exact  mathematical  equivalence  with  the  optimal  EFM  criterion  in  [5].  It  appears  that  using  the 
new  definitions  of  the  matrices  resulting  from  the  matrix  partitioning  in  (19),  the  AR- version  of  SMM  as  given 
in  (8)  can  also  be  decoupled  into  linear  and  nonlinear  subproblems.  This  equivalence  may  have  an  important 
consequence  for  the  proposed  algorithm.  There  already  exists  a  convergence  analysis  of  the  original  SM  method 
in  [47].  It  can  be  hoped  that  the  convergence  analysis  will  also  apply  to  the  decoupled  form  of  SM  method  given 
in  [34].  If  that  happens  to  be  the  case,  as  alluded  to  in  [34],  the  convergence  analysis  in  [47]  should  also  apply  to 
the  iterative  computational  algorithm  presented  in  this  work.  It  should  be  noted  though  that  the  results  of  SMM 
and  the  proposed  optimal  method  may  not  be  identical.  Specifically,  the  numerator  in  the  decoupled  form  of  [34] 
is  computed  somewhat  differently  than  (26)  which  is  the  optimal  estimate.  Furthermore,  it  should  be  also  added 
that  the  iterative  scheme  in  (35a)  is  not  the  only  possible  approach  for  iterative  minimization  of  the  equivalent 
criterion  in  (22).  In  fact,  removing  the  requirement  of  do  =  1,  the  eigenvector  corresponding  to  the  minimum 
eigenvalue  of  the  matrix  B^*))"1H ar  may  also  be  used  as  d^*+1),  the  estimate  at  the  (z-fil)-th  iteration 

step  [50].  This  possibility  is  not  obvious  from  the  original  SMM  algorithm  in  [3]. 

IV.  Simulation  Results 

In  this  Subsection,  the  performance  of  the  proposed  algorithm  is  evaluated  by  means  of  several  AR(p)  model 
identification  examples  with  different  p  values.  6  =  10” 6  was  used  as  the  stopping  criterion  in  both  phases  of  the 
algorithm  for  all  the  examples  below.  The  fitting-error  norm  defined  in  (5)  was  calculated  at  convergence  using 
the  estimated  parameters  and  the  results  are  tabulated  in  the  ‘Minimized  Error  Norm’  column.  Furthermore,  in 
order  to  get  a  relative  sense  of  performance,  the  logarithm  of  the  ratios  of  the  powers  of  the  ‘true  IR’  (known  in 
these  simulations)  and  the  error  powers  are  also  tabulated  in  the  ‘Closeness  in  dB’  columns. 

Simulation  1  : 

The  desired  impulse  response  has  a  Triangular  form  as  shown  by  the  solid  lines  in  Fig.  1A  -  ID.  The 
resulting  impulse  response  fit  using  Covariance  method  and  Auto-correlation  method  are  shown  as  connected 
circles  in  Figures  I A  and  IB,  respectively.  The  impulse  response  match  at  the  end  of  each  of  the  two  phases 
of  the  algorithm  described  in  Subsection  III  with  p  =  4  are  shown  in  Fig.  1C  and  Fig.  ID,  respectively.  The 
minimized  error  norm  and  the  closeness  of  the  fit  to  the  desired  signal  hp  are  listed  in  Table  1.  The  number  of 
iterations  for  convergence  are  also  listed.  It  can  be  seen  from  the  table  and  the  figures  that  compared  to  AM 
and  CM,  the  proposed  scheme  provided  more  accurate  estimates.  But  it  may  also  be  observed  that  there  is  no 
significant  difference  in  the  results  between  the  1st  and  the  2nd  phase  of  the  proposed  algorithm. 
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Table  1:  Example  1:  Comparison  of  three  methods  with  Triangular  Impulse  Response 


Method 

Closeness 

in  dB 

Minimized 
Error  Norm 

Number  of 
Iterations 

Covariance 

6.359 

154.9 

Auto-Correlation 

6.674 

144.093 

Proposed 

23.436 

3.037 

5 

Phase- 1 

Proposed 

23.439 

3.035 

3 

Phase-2 

Simulation  2  : 

An  arbitrary  impulse  response  was  generated  with  p  =  5  for  these  simulations.  If  the  algorithm  in  Subsection 
III  is  used  directly  to  match  the  true  response  it  will  give  perfect  results.  Instead,  Gaussianly  distributed  white 
noise  was  added  to  the  true  response  to  obtain  the  desired  response  h p.  Hence,  the  estimates  obtained  with  the 
proposed  algorithm  will  also  be  the  MLE  of  the  unknowns.  For  20dB  noise,  the  true  and  the  desired  responses 
are  shown  in  Fig.  2 A.  The  impulse  response  match  using  Covariance  method  and  Autocorrelation  method  are 
shown  in  Figures  2B  and  2C,  respectively.  The  initial  estimate  obtained  by  minimizing  the  equation  error  in 
(24a)  is  shown  in  Fig.  2D.  The  impulse  response  fit  obtained  using  the  proposed  algorithm  at  the  end  of  Phase- 1 
and  Phase-2  are  shown  in  Fig.  2E  and  Fig.  2F,  respectively.  The  minimized  error  norms  and  the  closeness  to  the 
true  response  are  listed  in  Table  2.  It  can  be  observed  for  this  example  that  there  is  about  3dB  difference  in  the 
impulse  response  fit  between  the  two  phases  though  the  difference  in  the  minimized  error  norms  is  quite  small. 
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Table  2  :  Example  2  :  Comparison  of  three  methods  with  5-th  order  Impulse  Response 


Method 

Closeness 
in  dB 

Minimized 
Error  Norm 

Number  of 
Iterations 

Covariance 

1.027 

1.847 

Auto-Correlation 

12.442 

0.691 

Equation  Error 

15.459 

0.656 

in  (24a) 

Proposed 

17.889 

0.646 

5 

Phase- 1 

Proposed 

20.883 

0.634 

4 

Phase-2 

From  these  simulation  results  a  fair  conclusion  may  be  drawn  that  the  Phase- 1  of  the  algorithm  does  an 
excellent  job  of  error  minimization.  Hence,  the  Phase-2  of  the  algorithm  need  not  be  invoked  for  most  applications. 
The  results  using  SMM  are  close  to  the  results  at  the  end  of  Phase- 1  if  the  original  SMM  [3]  or  the  decoupled 
form  in  [34]  are  used.  There  were  some  numerical  differences  in  the  coefficients  but  the  impulse  response  fit  looked 
almost  alike.  The  results  with  the  AR- version  of  SMM  given  in  [7]  were  poorer  than  Phase- 1  results  because  the 
numerator  coefficient  is  set  to  1  in  [7].  Extensive  simulations  with  other  examples  show  equivalent  performance. 
Interestingly,  the  simulations  also  indicate  that  the  proposed  algorithm  is  quite  immune  to  the  choice  of  initial 
estimates.  In  fact,  when  CM  or  AM  were  used  in  place  of  (32)  for  obtaining  the  initial  estimates,  the  results 
obtained  at  the  end  of  Phase- 1  or  Phase-2  turned  out  to  be  exactly  identical  to  the  results  listed  in  the  Tables.  But 
with  Covariance  method  as  initial  estimate,  the  Phase- 1  sometimes  took  one  or  two  extra  iterations  to  converge. 
This  important  observation  indicates  the  robustness  of  the  proposed  algorithm  to  the  choice  of  initial  estimates. 

V.  Concluding  Remarks 

In  this  Section,  a  classical  rational  model  identification  problem  has  been  addressed.  The  major  focus  was 
to  develop  an  algorithm  for  optimal  estimation  of  the  parameters  of  an  all-pole  transfer  function  with  arbitrary 
number  of  poles  by  model-fitting  a  prescribed  impulse  response.  Unlike  some  existing  results,  no  linearization 
or  approximation  has  been  done  while  deriving  the  theoretical  optimization  criterion.  It  is  shown  that  the 
multidimensional  non-linear  problem  can  be  decoupled  into  two  smaller  problems  of  which  one  is  a  linear  problem 
and  the  other  one  is  a  non-linear  problem.  The  inherent  mathematical  structure  of  the  non-linear  part  is  utilized 
to  formulate  an  efficient  iterative  computational  algorithm  for  estimating  the  denominator  parameters.  Global 
optimality  properties  of  the  estimators  have  been  confirmed  by  relating  the  multidimensional  optimization  problem 
to  certain  well-known  results  in  numerical  analysis.  In  simulation  studies  also,  the  method  has  been  shown  to 
be  highly  effective.  Regarding  possible  future  work,  it  may  be  noted  that  most  of  the  existing  suboptimal  1-D 
algorithms  have  been  extended  for  estimating  2-D  filter  coefficients  from  2-D  spatial  domain  data  [22,  29,  35, 
36,  40-43].  The  possibility  of  formulating  an  optimal  2-D  AR-filter  design  technique  by  extending  the  proposed 
method  is  being  studied  [49].  Extension  of  this  work  for  identification  of  Multidimensional  AR-systems  from 
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multidimensional  impulse  response  data  [38]  is  also  under  progress. 
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Fig.  1  :  Simulation  1  A  triangular  impulse  response  is  modeled  by  an  ABM  hi/ 

— - -  (c)  -  - 


KS'  2  ’  1  *~  AR(5)  ^  *»e  desired 

response  h Ffc~M  th  ““  “‘“d"'  The  &>«  denote  the  tree  impulse 

P  •  Fig.  2A,  the  connected  circles  show  the  noisy  signal  L  TntL  ti  , 

connected  circles  show  the  fit  with  (B)  Covariance  Method,  (Q  Autocorrelation  Method’  (D) 

minmuzation  of  Equation  error  in  (22f),  (E)  Phase-1  and  (F)  Phase-2  convergence.  ’ 


References 


[1]  R.  E.  Kalman,  “Design  of  a  Self  Optimizing  Control  System,”  Trans.  ASME,  Vol.  80,  pp.  468-478,  1958. 

[2]  J.L. Shanks,  “Recursion  Filters  for  Digital  Processing”,  Geophysics ,  Vol.  32,  pp.  33-51,  1967. 

[3]  K.  Steiglitz  and  L.E.  McBride,  “A  Technique  for  Identification  of  Linear  Systems”,  IEEE  Transactions  on 
Automatic  Control ,  Vol.  AC-10,  pp.  461-464,  1965. 

[4]  K.  Steiglitz,  “On  the  Simultaneous  Estimation  of  Poles  and  Zeros  in  Speech  Analysis,”  IEEE  Transactions 
on  Acoustics,  Speech  and  Signal  Processing ,  vol.  ASSP-25,  no.  3,  pp.  229-234,  June,  1977. 

[5]  A.G.  Evans  and  R.  Fischl,  “Optimal  Least  Squares  Time-Domain  Synthesis  of  Recursive  Digital  Filters”, 
IEEE  Transactions  on  Audio  and  Electro-Acoustics ,  Vol.  AU-21,  pp.  61-65,  1973. 

[6]  A.  K.  Shaw,  “Optimal  Identification  of  Discrete-Time  Systems  From  Impulse  Response  Data,”  To  be  pub¬ 
lished,  IEEE  Transactions  on  Acoustics,  Speech  and  Signal  Processing,  Jan.,  1994. 

[7]  L.B.  Jackson,  Digital  Filters  and  Signal  Processing ,  Kluwer,  Boston,  1986. 

[8]  J.  Makhoul,  “Linear  Prediction  :  A  Tutorial  Review,”  Proceedings  of  the  IEEE ,  vol.  63,  pp.  561-580,  April, 
1975. 

[9]  S.M.  Kay,  Modern  Spectral  Estimation:  Theory  and  Applications ,  Prentice  Hall,  Englewood  Cliffs,  NJ,  1988. 

[10]  L.  L.  Scharf,  Statistical  Signal  Processing  -  Detection,  Estimation  and  Time  Series  Analysis ,  Addison- Wesley, 
Reading,  MA,  1990. 

[11]  J.  Durbin,  “Efficient  Estimation  of  Parameters  in  Moving- Average  Models,”  Biometrika ,  vol.  46,  pp.  306-316, 
1959. 

[12]  A.  Stefanski  and  C.  Weygandt,  “Extension  of  the  Steiglitz  and  McBride  Identification  Technique,”  IEEE 
Transactions  on  Automatic  Control ,  pp.  503-504,  Oct.,  1971. 

[13]  R.  Kumaresan,  L.  L.  Scharf  and  A.  K.  Shaw,  “An  Algorithm  for  Pole-Zero  Modeling  and  Spectral  Estimation,” 
IEEE  Transactions  on  Acoustics  Speech  and  Signal  Processing ,  vol.ASSP-34,  pp.  637-640,  June,  1986. 

[14]  L.  Ljung,  System  Identification:  Theory  for  the  Users ,  Prentice  Hall,  NJ,  1987. 

[15]  C.  S.  Burrus,  T.  W.  Parks,  and  T.  B.  Watt,  “A  Digital  Parameter-Identification  Technique  Applied  to 
Biological  Signals,”  IEEE  Trans,  on  Bio-Medical  Engineering ,  vol.  BME-18,  pp.  35-37,  Jan.,  1971. 

[16]  Y.  Bressler  and  A.  Macovski,  “Exact  Maximum  Likelihood  Parameter  Estimation  of  Superimposed  Expo¬ 
nential  Signals  in  Noise,”  IEEE  Transactions  on  Acoustics,  Speech  and  Signal  Processing ,  vol.  ASSP-34,  no. 
10,  pp.  1081-1089,  Oct.,  1986. 

[17]  T.  Soderstrom  and  P.  Stoica,  System  Identification ,  Prentice  Hall,  NJ,  1987. 

[18]  G.  H.  Golub  and  V.  Pereyra,  “The  Differentiation  of  Pseudoinverses  and  Nonlinear  Problems  Whose  Variables 
Separate,”  SIAM  Journal  on  Numerical  Analysis ,  vol.  10,  no.  2,  pp.  413-432,  Apr.,  1973. 

[19]  H.  D.  Scolnik,  “On  the  Solution  of  Nonlinear  Least  Squares  Problems,”  Proceedings  of  IFIP-1971,  Numerical 
Mathematics,  North  Holland,  Amsterdam,  pp.  18-23,  1971. 

[20]  M.  R.  Osborne,  [1975],  “Some  Special  Nonlinear  Least  Squares  Problems,”  SIAM  Journal  on  Numerical 
Analysis ,  vol.  12,  no.  4,  pp.  571-592,  Sep.,  1975. 


116 


[21]  I.  Guttman,  V.  Pereyra,  “Least  Squares  Estimation  for  a  Class  of  Non-Linear  Models,”  Technometrics ,  vol. 
15,  no.  2,  pp.  209-218,  May,  1973. 

[22]  J.  S.  Lim,  Two-Dimensional  Signal  and  Image  Processing ,  Prentice-Hall,  Englewood  Cliffs,  NJ,  1980. 

[23]  A.  K.  Jain,  “Fast  Inversion  of  Banded  Toeplitz  Matrices  by  Circular  Decompositions,”  IEEE  Transactions 
on  Acoustics ,  Speech  and  Signal  Processing ,  vol.  ASSP-26,  no.  2,  pp.  121-126,  April,  1978. 

[24]  E.  R.  Schulz,  “Estimation  of  Pulse  Transfer  Function  Parameters  by  Quasilinearization,”  IEEE  Transactions 
on  Automatic  Control,  pp.  424-426,  1968. 

[25]  K.  Steiglitz,  “Computer-Aided  Design  of  Recursive  Digital  Filters,”  IEEE  Transactions  on  Audio  and  Elec¬ 
troacoustics,  vol.  AU-18,  no.  2,  pp.  123-129,  June,  1970. 

[26]  A.  G.  Deczky,  “Synthesis  of  Recursive  Digital  Filters  Using  the  Minimum  p- Error  Criterion,”  IEEE  Trans¬ 
actions  on  Audio  and  Electroacoustics ,  vol.  AU-20,  no.  4,  pp.  257-263,  June,  1970. 

[27]  R.  Fischl,  “Optimal  lp -Approximation  of  Prescribed  Impulse  Response  Functions  on  a  Finite  Point  Set,”  in 
Proc.  IEEE  Int .  Symp.  on  Circuit  Theory ,  pp.  155-156,  1970. 

[28]  F.  Brophy  and  A.  C.  Salazar,  “Recursive  Filter  Synthesis  in  the  Time  Domain,”  IEEE  Transactions  on 
Acoustics,  Speech  and  Signal  Processing,  vol.  ASSP-22,  no.  1,  pp.  45-55,  Feb.,  1974. 

[29]  M.  S.  Bertran,  “Approximation  of  Digital  Filters  in  One  and  Two  Dimensions,”  IEEE  Transactions  on 
Acoustics,  Speech  and  Signal  Processing,  vol.  ASSP-23,  no.  5,  pp.  438-443,  Oct.,  1975. 

[30]  C.  S.  Burrus  and  T.  W.  Parks,  “Time  Domain  Design  of  Recursive  Digital  Filters,”  IEEE  Transactions  on 
Audio  and  Electroacoustics ,  vol.  AU-18,  pp.  137-141,  June,  1970. 

[31]  C.  Charalambous,  “Minimax  Optimization  of  Recursive  Digital  Filters  Using  Recent  Minimax  Results,”  IEEE 
Transactions  on  Acoustics,  Speech  and  Signal  Processing,  vol.  ASSP-23,  no.  4,  pp.  333-345,  June,  1977. 

[32]  M.  J.  Levin,  “Estimation  of  a  System  Pulse  Transfer  Function  in  the  Presence  of  Noise,”  IEEE  Transactions 
on  Automatic  Control ,  pp.  229-235,  1964. 

[33]  G.  Miller,  “Least-Squares  Rational  Z-Transform  Approximation,”  Journal  of  the  Franklin  Institute,  vol.  295, 
no.  1,  pp.  1-7,  Jan.,  1973. 

[34]  J.  H.  McClellan  and  D.  Lee,  “Exact  Equivalence  of  the  Steiglitz-McBride  Iteration  and  IQML,”  IEEE  Trans¬ 
actions  on  Acoustics,  Speech  and  Signal  Processing,  vol.  ASSP-39,  no.  2,  pp.  509-512,  Feb.,  1991. 

[35]  J.L.  Shanks,  S.  Treitel  and  J.H.  Justice,  “Stability  and  Synthesis  of  Two-Dimensional  Recursive  Filters” 
IEEE  Transactions  on  Audio  and  Electroacoustics ,  Vol.  AU-20,  pp.  115-128,  1972. 

[36]  J.  A.  Cadzow,  “Recursive  Digital  Filter  Synthesis  via  Gradient  Based  Algorithms”,  IEEE  Transaction  on 
Acoustic,  Speech  and  Signal  Processing ,  Vol.  ASSP-24,  pp.  349-355,  1976. 

[37]  A.  K.  Shaw  and  P.  Misra,  “Time  Domain  Identification  of  Proper  Discrete  Systems  from  Measured  Impulse 
Response  Data,”,  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing ,  pp.  1689-1692, 
Toronto,  Canada,  May,  1991. 

[38]  A.  K.  Shaw,  P.  Misra  and  R.  Kumaresan,  “Identification  of  Multi-Dimensional  Systems  From  Impulse  Re¬ 
sponse  Data,”  30th  IEEE  Conf  on  Dec.  and  Contr .,  Brighton,  UK,  Dec.  1991. 

[39]  C.  R.  Rao  and  S.  K.  Mitra,  Generalized  Inverse  of  Matrices  and  its  Applications.  New  York,  Wiley,  1971. 


117 


[40]  D.E.  Dudgeon  and  R.M.  Mersereau,  Multidimensional  Digital  Signal  Processing ,  Englewood  Cliffs,  NJ,  Pren¬ 
tice  Hall,  1984. 

[41]  G.  A.  Shaw  and  R.  M.  Mersereau,  “Design,  Stability  and  Performance  of  Two-Dimensional  Recursive  Digital 
Filters”,  Tech.  Report  E21-B05-1 ,  Georgia  Inst,  of  Technology  School  of  Electrical  Engg.,  1979. 

[42]  T.  Hinamoto  and  S.  Maekawa,  “Separable- Denominator  2-D  Rational  Approximation  via  1-D  Based  Algo¬ 
rithm  ”,  IEEE  Transactions  on  Circuits  and  Systems ,  Vol.  CAS-32,  pp.  989-999,  Nov.,  1985. 

[43]  T.  Hinamoto,  “Design  of  2-D  Separable-Denominator  Recursive  Digital  Filters”,  IEEE  Transactions  on 
Circuits  and  Systems ,  Vol.  CAS-31,  pp.  925-933,  Nov.,  1984. 

[44]  R.  Kumaresan  and  A.  K.  Shaw,  “High  Resolution  Bearing  Estimation  Without  Eigendecomposition,”  Pro¬ 
ceedings  of  IEEE  International  Conference  on  Acoustics  Speech  and  Signal  Processing ,  Florida,  April,  1985. 

[45]  A.K.  Shaw  and  R.  Kumaresan,  “Some  Structured  Matrix  Approximation  Problems”,  Proc.  of  the  IEEE 
International  Conference  on  Acoustics,  Speech  and  Signal  Processing ,  New  York,  NY,  pp.  2324-2327,  April, 
1988. 

[46]  R.  Kumaresan  and  A.K.  Shaw,  “Superresolution  by  Structured  Matrix  Approximation”,  IEEE  Transactions 
on  Antennas  and  Propagation ,  Vol.  AP-36,  pp.  34-44,  Jan.,  1988. 

[47]  P.  Stoica  and  T.  Soderstrom,  “The  Steiglitz-McBride  Identification  Algorithm  Revisited  -  Convergence  Anal¬ 
ysis  and  Accuracy  Aspects,”  IEEE  Transactions  on  Automatic  Control ,  vol.  AC-26,  no.  3,  pp.  712-717, 
June,  1981. 

[48]  E.  Bodewig,  Matrix  Calculus ,  North-Holland  Publishing  Co.,  Amsterdam,  1956. 

[49]  A.  K.  Shaw,  “Optimal  Design  of  Two-Dimensional  Filters  from  Spatial  Domain  Data,”  under  progress. 

[50]  K.  Hoffman  and  R.  Kunze,  Linear  Algebra ,  Prentice  Hall,  Englewood  Cliffs,  New  Jersey,  1971. 


APPENDIX  Computational  Algorithm  :  Phase  II 


The  second  phase  of  the  iterative  algorithm  is  described  in  this  Appendix.  In  this  phase,  the  derivative  of  the 
matrix  W  w.r.t.  b  is  taken  into  consideration  while  minimizing  the  fitting  error  norm.  The  complete  expression 
of  the  ^2-norm  of  the  error  can  be  written  as, 


||e|||  =  eTe  =  (Wg  +  WGd)T(Wg  +  WGd). 


(Al) 


By  setting  the  derivative  of  this  squared  norm  to  zero, 


the  updated  at  the  ( i  +  l)-th  iteration  is  given  by, 


b(«+i)  _  _  [u(‘)G]_1[U(*)]g 


where  (suppressing  the  superscript  (’)), 
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has  the  same  form  as  the  B  matrix  defined  in  (19a)  but  filled  with  all  zeros  except  at  the  locations  where  dk 
appear.  For  example, 
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Once  b(,+1)  is  found,  b(,+1)  can  be  formed  as, 
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(A.36) 


This  minimization  phase  continues  until  b,+1  ~  b‘  is  reached  and  this  optimum  b"  vector  corresponds  to  a 
minimum  of  the  error  surface  of  ||e||2. 
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Section  -  3.4  :  DESIGN  OF  DENOMINATOR-SEPARABLE  2-D  HR  FILTERS 


SUMMARY 

Optimal  design  of  an  important  class  of  two-dimensional  (2-D)  digital  IIR  filters  from  spatial  impulse  response 
data  is  addressed.  The  denominator  of  the  desired  2-D  filter  is  assumed  to  be  separable  into  two  1-D  factors. 
The  filter  coefficients  are  estimated  by  minimizing  the  ^-norm  of  the  error  between  the  prescribed  and  the 
estimated  spatial  domain  responses.  The  denominator  and  numerator  estimation  problems  are  theoretically 
decoupled  into  separate  problems.  The  decoupled  criteria  have  reduced  dimensionality.  The  denominator  criterion 
is  simultaneously  optimized  w.r.t .  the  coefficients  in  both  dimensions  using  an  iterative  algorithm.  The  numerator 
coefficients  are  found  in  a  straight-forward  manner.  If  the  desired  response  is  known  to  be  symmetric,  the 
proposed  algorithm  can  be  constrained  to  produce  optimal  denominators  which  are  identical  in  both  domains. 
The  performance  of  the  algorithm  is  demonstrated  with  simulation  examples. 

I.  Introduction 

Two-Dimensional  IIR  filters  are  commonly  used  in  image  processing  and  2-D  filtering.  Synthesis  of  such 
filters  from  prescribed  spatial  domain  impulse  response  data  is  an  important  and  challenging  design  problem  and 
has  received  considerable  attention  in  recent  literature  [1,  2,  4,  5,  8,  12,  14,  16].  Spatial-domain  design  of  2-D 
IIR  filters  is  analogous  to  1-D  recursive  filter  design  based  on  time-domain  specifications.  Most  2-D  filter  design 
algorithms  are  basically  extensions  of  existing  1-D  algorithms.  In  particular,  Shanks  et  al  [12]  had  extended  the 
work  of  Shanks  [11];  Cadzow  [1]  and  Shaw  and  Mersereau  [16]  utilized  many  of  the  general  non-linear  optimization 
methods;  and  Shaw  and  Mersereau  [16]  also  extended  the  work  of  Steiglitz  and  McBride  [17].  The  1-D  work  of 
Mullis  and  Roberts  [7]  was  further  extended  and  applied  to  the  2-D  case  in  [5]. 

The  approaches  noted  above  do  not  minimize  the  true  spatial  impulse  response  error,  though  it  may  be 
mentioned  that  the  extension  of  Steiglitz-McBride  method  in  [16]  closely  approximates  the  true  fitting  error.  For 
the  strictly-proper  case,  i.e.,  when  the  numerator  order  is  one  less  than  that  of  the  denominator,  Evans  and  Fischl 
(EFM)  had  proposed  an  optimal  method  for  synthesis  of  1-D  IIR  filters  [3].  The  2-D  filter  synthesis  algorithm 
presented  here  is  a  generalization  of  EFM  to  2-D.  Proposed  solution  is  optimal  in  the  sense  that  it  minimizes  true 
and  complete  spatial  error  criterion  for  the  design  of  strictly-proper  2-D  IIR  filters. 

EFM  has  been  found  to  be  highly  accurate  for  1-D  filter  design.  A  modified  complex  version  of  the  EFM 
with  certain  symmetry  constraints  has  also  been  shown  to  be  effective  for  maximum-likelihood  1-D  and  2-D 
frequency- wavenumber  estimation  [  8,  13,  15].  Generalization  of  EFM  for  strictly-proper  2-D  filter  design  has  also 
been  considered  previously  [4,  5],  but  it  appears  that  the  full  potential  of  EFM  has  not  been  utilized  in  the  2-D 
case.  Specifically,  it  will  be  shown  that  the  complete  error  criterion  encompassing  the  entire  subspace  orthogonal 
to  the  model  fitting  error  was  not  optimized  in  [4,  5].  Instead,  two  suboptimal  error  criteria  were  formed  in  each 
domain  and  the  filter  coefficients  were  optimized  in  the  two  dimensions  independently. 

In  this  Section,  a  2-D  version  of  EFM  is  developed  for  optimal  design  of  2-D  recursive  filters  from  prescribed 
spatial  domain  data.  The  complete  basis  space  orthogonal  to  the  spatial  fitting  error  will  be  identified  and  the 
corresponding  error  criterion  will  be  shown  to  be  dependent  only  on  the  2-D  filter  parameters.  Similar  to  1-D 
EFM,  the  non-linear  error  criterion  will  be  decoupled  into  a  purely  linear  and  a  non-linear  sub-problem.  For 
the  separable  denominator  case,  it  is  also  shown  that  the  error  vector  possesses  a  quasi-linear  relationship  with 
the  denominator  coefficients  in  both  domains  simultaneously.  Unlike  several  existing  2-D  methods  [1,  3,  4,  5], 
the  exact  fitting  error  is  minimized  w.r.t.  the  filter  coefficients  in  both  dimensions  simultaneously.  Simultaneous 
optimization  is  particularly  effective  for  synthesizing  2-D  filters  with  symmetric  impulse  response  which  are  quite 
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common  in  practice.  In  such  cases,  the  criterion  can  be  constrained  to  produce  identical  denominators  in  both 
domains  ensuring  symmetry  in  the  estimated  spatial  response. 

The  Section  is  arranged  as  follows  :  In  Subsection  II,  the  least-squares  problem  is  stated.  In  Subsection  III, 
the  preliminaries  for  the  non-separable  numerator  and  separable  denominator  case  is  given.  In  Subsection  IV, 
the  new  orthogonal  basis  spaces  are  defined,  the  error  criterion  is  derived  and  the  computational  algorithm  is 
summarized.  In  Subsection  V,  some  simulation  results  are  given  and  finally,  in  Subsection  VI  some  concluding 
remarks  along  with  directions  for  future  work  are  included. 

II.  Problem  Statement  and  Formulation 

In  general,  a  2-D  rational  function  H(z\}  ^2),  with  non-decomposable  numerator  and  denominator  is  described  as 
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Given  the  2-D  spatial  impulse  response  matrix  X,  estimate  p  and  q  by  optimizing  the  following  error  criterion, 


min||e||2  A  ||x  —  h||2  with,  p(0,0)  =  1,  where, 
q,p  = 

(7a) 

q  A  [g(0,0)  g(0, 1)  •••  q(n1,n2)]T  and 

(76) 

p  A  [p(0, 0)  p(0, 1)  •••  p(m1,m2)]T 

(7c) 

This  problem  is  nonlinear  in  p  and  standard  gradient-based  optimization  algorithms  have  been  used  for  1-D  as 
well  as  for  2-D  designs  [1,  2,  16].  But  these  generic  algorithms  do  not  make  effective  use  of  the  matrix-structures 
inherent  in  this  particular  problem  and  they  are  known  to  be  sensitive  to  initialization.  Several  sub-optimal 
algorithms  based  on  linearization  of  the  true  non-linear  criterion  have  also  been  proposed  [2,  11,  16,  17].  In  this 
work,  the  exact  fitting  criterion  defined  in  (7)  will  be  theoretically  decoupled  into  a  purely  linear  problem  for  q 
and  a  non-linear  problem  for  p.  Furthermore,  the  non-linear  criterion  will  be  shown  to  possess  a  quasi-linear 
relationship  to  the  unknown  denominator  coefficients.  This  will  lead  to  the  formulation  of  an  iterative  algorithm 
for  its  minimization. 


III.  Design  With  Separable  Denominator  and  Non- Separable  Numerator 

In  this  case,  the  2-D  rational  transfer  function  can  be  written  as, 

H(z  z)  =  E TJo'TZZ'Mi)*?* 
u  2  Er=loc(OTEr=VO>2-J'  ' 

Define, 

c  A  [c(0)  c(l)  ...c(mi)]T  and 

d  A  [d(0)  d(l)  ...d(m2)]T. 


(8a) 


(86) 

(8c) 


Multiplying  both  sides  of  (8a)  by  EI*Lo  c(*)zi  *  Ej^o  ^U)z2  3  anc*  equating  the  coefficients  of  the  same  powers  of 

Z"X  [5], 


[Df  ®  cf]h  =  q  (9a) 

[Df  ®  CT]h  =  0  (96) 

[Dt  ®  C^]h  =  0  (9c) 

[Dt  <8>  CT]h  =  0  (9 d) 

[Ifcs  ®  CT]h  =  0  (9e) 

[Dt  ®  Ifc,]h  =  0  where,  (9/) 


c(mi) 

0 

0 

c(mi  —  1) 

c(mi)  •  •  • 

0 

c(  0) 

c(l)  ••• 

c(mi) 

0 

c(0)  ••• 

c(mi  —  1) 

0 

0 

m  . 
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d(m2) 

0 

0 

d(m2 

-1) 

d(m2)  •  •  • 

0 

d( 0) 

d(l)  • 

d(m2) 

E  JR^2  ^(^2  —  m2) 

(106) 

0 

d(  0)  •• 

d{m2  -  1) 

0 

0 

d(  0) 

rc(0) 

c(l)  ••• 

c(mi  —  1) " 

0 

c(0)  • •  • 

c(mi  —  2) 

4  A 

0 

0  ••• 

C(  0) 

el1'*1, 

(10c) 

0 

0  ••• 

0 

.  0 

0  ••• 

0 

rd(0) 

d(  1)  ••• 

d(m2  —  1)' 

0 

d(  0) 

d[m2  —  2) 

*1  A 

0 

0  ••• 

d{ 0) 

e  jR*2Xm2) 

(10  d) 

0 

0  ••• 

0 

.  0 

0  ••• 

0 

where,  1*^  E  TRklXkl  and  Ifc2  E  JRk2Xk2  are  identity  matrices  and  0  denotes  the  Kronecker  product  [9].  If  h,  c 
and  d  are  known,  the  numerator  vector  q  can  be  calculated  using  (9a).  But  in  practice,  h,  c  and  d  need  to  be 
estimated  from  the  prescribed  response  x.  If  h  is  replaced  by  the  prescribed  x  in  (9b)-(9f),  the  right  hand  sides 


will  not  be  equal  to  zero.  Instead,  it  will  result  in  the  following  equation  errors  : 

[D[  0  CT]x  =[Df  0  CT][h+  e]  =  [Df  0  CT]e  A  e\q  (11a) 

[DT0Cf]x^DT0Cf][h  +  e]  =  [DT0Cf]e  A  e2eq  (116) 

[Dt  0  CT]x  =[Dt  0  CT][h  +  e]  =  [Dt  0  CT]e  A  (11c) 

[I*3  <g>  CT]x  =[I*2  (g.  CT][h  +  e]  =  [Ifc2  ®  CT]e  A  (lid) 

[Dt  <g>  Jfcjx  =[Dt  ®  4,1th  +  e]  =  [Dt  ®  4,]e  A  e\q.  (lie) 


In  (lla)-(lle),  the  fact  that  x  =  h  +  e  and  the  orthogonal  relationships  in  (9b)-(9f)  have  been  utilized.  These 
equations  show  the  relationships  between  the  fitting  error  e  and  equation  errors.  As  in  case  of  1-D  EFM  [7],  in 
order  to  minimize  ||e||2  an  inverse  relationship  of  the  form, 

e  =  W(c,d)ee9  (12) 


need  to  be  found.  The  matrix,  W(c,  d)  needs  to  be  constructed  using  c  and  d. 

The  problem  of  determining  the  denominator  coefficient  vectors  c  and  d  is  essentially  equivalent  to  the 
search  for  (kik2  —  ^1^2)  linearly  independent  vectors  orthogonal  to  h.  These  orthogonal  basis  space  must  be 
dependent  on  the  elements  in  c  and  d  only.  Equation  (9a)  clearly  shows  that  Dx  0  Ci  E  k\k2  x  mim2  can  not  be 
orthogonal  to  h.  On  the  other  hand,  (9b)-(9f)  demonstrate  that  the  matrices  Di0C,  D0Ci,  D0C,  I*20C  and 
DT  0  7jk1  are  indeed  all  orthogonal  to  h.  But  summing  the  respective  number  of  columns  of  these  five  full-rank 
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matrices,  the  total  turns  out  to  be  (3fci^2  —  m\ m?  —  kxm^  —  A^mi)  vectors  of  length  k\k2  each.  Hence,  these  set 
of  matrices  can  not  all  be  linearly  independent  of  each  other.  In  [5],  the  matrices,  Di  ®  C,  D  ®  Ci,  D  ®  C,  as 
formed  in  (9b)-(9d),  respectively,  were  utilized  to  form  an  inverse  relationship  as  required  by  (12).  These  set  of 
matrices  do  contain  {k\k^  —  mi  m2)  linearly  independent  vectors  of  length  &i&2,  but  unfortunately,  they  are  not 
orthogonal  to  each  other. 

It  may  be  noted  here  that,  in  a  previous  2-D  generalization  of  EFM  [4],  the  complete  spatial  fitting  error 
criterion  was  not  formulated.  In  a  later  work,  it  was  partially  formulated  in  equation  (14)  of  [5],  but  no  algorithm 
was  presented  for  minimizing  that  criterion  with  respect  to  the  unknown  parameters.  Instead,  in  both  those 
works,  two  separate  criteria  were  minimized  independently.  Specifically,  (9e)  and  (9f)  were  used  in  [4,  5]  to 
estimate  c  and  d  using  two  independent  optimizations.  But  Ijfe2  ®  C  and  D  ®  contain  (&i&2  —  ^2^1)  and 
(&i&2  —  m2&i)  linearly  independent  vectors,  respectively.  The  entire  (&i&2  —  mim2)  dimensional  vector-space 
orthogonal  to  h  was  not  optimized  simultaneously  w.r.t  c  and  d.  It  is  not  apparent  if  the  optima  of  these  separate 
criteria  are  identical  to  those  of  the  true  2-D  criterion.  In  Subsection  IV,  a  new  set  of  orthogonal  vectors  will  be 
constructed  which  will  lead  to  the  formulation  of  an  exact  2-D  spatial  fitting  error  criterion  that  can  be  optimized 
simultaneously  w.r.t.  c  and  d. 

IV.  Formulation  of  the  Orthogonal  Basis  Space  : 

According  to  orthogonality  principle  [15],  the  fitting  error  e,  at  minimum,  ought  to  be  orthogonal  to  the 
‘estimated’  h  that  minimizes  the  error.  It  is  also  desirable  to  have  the  resulting  error  criterion  dependent  on  the 
denominator  coefficients  only.  To  meet  these  requirements  two  Vandermonde  matrices  are  formed  as  follows, 


"  1 

1  ' 

■  1 

1  ■ 

^1 

tmi 

?i 

T  A 

t>  . 

t* 

m  1 

and  Q  A 

9?  • 

lll 

tki~ 1 

•  *  Lm  1  J 

1 - 

?T* 

W  .  . 

1 

where,  t{  —  eJ’Wi;u =  l,...,mi  and  g*  =  i  =  l,...,m2  be  the  roots  of  the  polynomials  C(z\)  = 


J2T=o  ci})z  1  *  anc*  d(0z2  \  respectively.  Hence,  by  construction, 

CtT  =  0  and  (14a) 

DtQ  =  0  (146) 

Furthermore,  using  (9e)  and  (9f), 

[QT  ®  CT]h  =  [Qt  ®  I][I®  CT]h  =  0  and  (15a) 

[Dr<g>TT]h  =  [I  ®  Tt][Dt  <g>  I]h  =  0.  (156) 


The  orthogonality  relationships  in  (9d),  (15a)  and  (15b)  demonstrate  that  the  three  matrices  Q  ®  C,  D  ®  T 
and  D  ®  C  together  constitute  (&i&2  —  ^1^2)  dimensional  vector  space  orthogonal  to  h.  Interestingly,  these 
matrices  are  not  only  formed  with  linearly  independent  columns  they  are  also  mutually  orthogonal  to  each  other. 
This  orthogonality  claim  can  be  easily  substantiated  as  follows  : 

[QT(gCT][D(g>T]  =  [QtD®CtT]  =  [0®0]  =  0, 

[Qt  0  Ct][D  ®  C]  =  [QTD(g)CTC]  =  [0(g)  CTC]  =  0 
[Dt®Tt][D®  C]  =  [DrD(g)TTC]  =  [DTD  ®  0]  =  0. 
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and 


(16a) 

(166) 

(16c) 


It  should  also  be  mentioned  here  that  the  matrices  T  and  Q  are  useful  only  in  this  intermediate  stage  of  deriving 
the  2-D  error  criterion  and  they  will  not  be  needed  in  the  final  optimization  steps. 

If  the  vector  h  in  (15)  is  replaced  by  x,  then  similar  to  (11),  the  following  equation  errors  are  formed  : 

/Qt0Ct\  /  qt  0  CT  \ 

Dr®TT  x=  Dt®Tt  e  A  eer  (17) 

\dt®ct/  \dt®ct/  — 

The  optimized  e  must  be  orthogonal  to  the  estimated  h,  whereas  the  three  matrices,  Q  0  C,  D  0  T  and  D  0  C, 
were  shown  to  be  orthogonal  to  h  in  (9d),  (15a)  and  (15b),  respectively.  Hence,  e  can  be  constructed  as  a  linear 
combination  of  the  columns  of  these  orthogonal  set  of  matrices,  i.e., 

e  =  (Q0C  D  0  T  D  0  C  )f  (18) 

where, 

f  A  [/i  f2  •••  fk1ki-m1m3]T  (19) 

is  a  vector  of  constants  which  are  to  be  determined.  Using  this  form  of  e  in  (17),  the  equation  error  can  be  written 
as, 

/Qt0Ct\  /QtQ0CtC  0  0  \ 

Dt0Tt  x=  0  DtD0TtT  0  f  A  ee?,  (20) 

\Dt0Ct/  \  0  0  DtD0CtC/  — 

The  matrix  on  the  r.h.s.  is  square  block-diagonal  with  square  diagonal  blocks  and  hence  it  can  be  inverted  to 
uniquely  determine  the  vector  of  constants  f  as, 


/(QtQ)-1®(CtC)-1 

0 

0  \ 

/qt®ct\ 

0 

(DtD)-10(TtT)-1 

0 

Dt  ®  Tt  X  (21a) 

V  0 

0 

(DtD)-1®(CtC)-1/ 

\Dt®Ct/ 

/(QtQ)-1Qt0(CtC)-1Ct\ 

=  (DtD)-1Dt0(TtT)"1Tt  X.  (216) 

\  (dtd)*“1dt  0  (cTcy1T>T  ) 

Using  this  expression  of  f  in  (18)  the  fitting  error  becomes, 

e  =  [Pq07)c+^d0Pt+7,d®  Pc]x  (22) 

where,  V[.]  denotes  the  Projection  matrix,  e.</.,  Vc  A  C(CrC)“1CT.  Unfortunately,  this  error  vector  is 
dependent  on  T  and  Q  which  must  be  removed.  According  to  (14),  the  matrices  C  and  D  are  orthogonal  to  T 
and  Q,  respectively.  Furthermore, 


rank(T)  +  rank(C)  =  k\  and  (23a) 

rank(Q)  +  rank(D)  =  k<i  (236) 

Hence,  using  a  theorem  on  Projection  matrices  [14], 

Vc  +  Vt  — Ijbi  and  (24a) 

Vu  +  VQ  =I*2.  (246) 
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Using  these  relationships  in  (22),  Vt  and  Vq  can  now  be  replaced  and  the  error  vector  can  be  written  as, 


e  =  [(I*3  -  Vn)®Vc  +  7>D®(Ifcx  -  Vc)  +  ?D®^c]x  (25a) 

=  [h2®Vc  ~  Vn®Vc  +  Pd®  I*  -  Vb®Vc  +  Vn®Vc]x  (256) 

=  [I k2®Vc  +  Pd  ®  Ifci  —  V^o®V c]x.  (25c) 

Note  that  in  this  final  form  of  the  error,  there  is  no  dependence  on  either  T  or  Q.  Hence,  the  error  criterion  for 
determining  the  denominator  coefficient  vectors  c  and  d  can  now  be  written  as, 

mine||(c,  d)| | 2  =  min  (xT[Ijk3  0  Vc  +  Pd  ®  Ikl  ~  Pd  ®  Pc]x)  (26) 

c,d  c,d 


Equations  (26)  and  (9a)  represent  the  desired  decoupled  criteria  for  determining  the  denominator  and  numerator 
coefficients,  respectively.  Optimization  of  (26)  would  produce  the  optimal  c  and  d,  denoted  as,  c°  and  d°, 
respectively.  Letting  e°  denote  the  minimized  error  corresponding  to  the  optimum  denominator  coefficients,  the 
optimum  spatial- response  vector  h  can  be  found  from, 


h°  A  x  —  e°. 


(27) 


This  h°  can  then  be  used  in  (9a)  to  obtain  the  optimal  numerator  vector,  q° . 

Analyzing  the  criteria  in  (26)  it  is  apparent  that  the  first  two  terms  are  the  orthogonal  projections  of  the  data 
x  on  to  the  parameter  spaces  of  each  of  the  two  spatial  dimensions.  The  third  term  is  the  orthogonal  projection 
common  to  both  dimensions  but  is  subtracted  once  because  the  common  (or,  joint)  projections  have  already  been 
included  once  in  each  of  the  first  two  terms.  It  is  very  interesting  to  note  that  this  criterion  is  quite  analogous 
to  the  standard  formula  of  the  Probability  of  Union  of  two  subsets.  It  may  be  emphasized  here  that  this  form  of 
the  error  criterion  is  not  only  mathematically  appropriate  it  is  intuitively  appealing  as  well  and  this  form  of  the 
2-D  error  criterion  was  not  arrived  at  in  any  of  the  previous  generalizations  of  EFM  [4,  5].  With  further  algebraic 
manipulations,  the  error-vector  e  can  also  be  shown  to  be  related  to  both  the  denominator  vectors  c  and  d  in  a 
quasi- linear  manner  as, 

e(c,  d)  A  ( (I*3  —  Pd)  <8>  W(c))X1  (W(d)  ®  I*,)X2  )  ( J )  ,  (28a) 

where, 


W(c)  A  C(CTC)_1  and 
W(d)  A  D(DtD)_1. 


X1  and  X2  are  constructed  from  the  elements  in  X  as  [9,  17], 


where,  the  (/,  k)th  term  of  Xt-  is  formed  as, 


X2  A 


(XT  xf 

••• 

+  n»i  +  1), 

for  i  =  1,2. 

^Xm24-1 

xi  \ 

XTO2+2 

x2 

V  Xjfcj 

xA*2-m2  ' 

(286) 

(28c) 

(29) 

(30a) 

(306) 
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Equation  (28a)  is  one  the  key  results  derived  in  this  Section.  It  clearly  shows  that  both  the  unknown  coefficient 
vectors  c  and  d  appear  simuitaneously  in  a  ‘quasi- linear’  relationship  w.r.t.  the  true  error  vector.  This  quasi-linear 
relationship  can  be  exploited  for  simultaneous  optimization  of  the  criterion  in  (26)  w.r.t.  c  and  d.  The  algorithm 
is  similar  in  flavor  to  the  ones  in  [3-5,  8,  13],  except  that  both  the  denominators  are  optimized  and  estimated 
simultaneously.  Specifically,  the  algorithm  iteratively  minimizes  the  ^-norm  of  the  error  vector  formed  in  (26) 
in  two  phases.  In  Phase- 1,  the  W  matrices  are  treated  as  constants  and  are  formed  using  the  estimates  of  c  or 
d  obtained  at  the  previous  iteration.  In  Phase-2,  the  estimates  are  improved  upon  by  setting  the  gradient  of  the 
complete  error-norm  to  zero.  The  iterations  are  initialized  by  setting  c  =  [1  0  ...  0]T  and  d  =  [1  0  ...  0]T. 
The  iterations  are  continued  until  the  changes  in  the  estimates  in  successive  iterations  become  very  small.  It  may 
be  noted  here  that  extensive  simulation  experience  in  1-D  [7,  8,  13,  14]  as  well  as  for  2-D  cases  [13,  15]  indicate 
that  Phase- 1  itself  produces  very  good  estimates  of  the  filter  coefficients  and  in  most  cases,  there  may  not  be  any 
need  for  invoking  Phase-2  at  all.  It  may  be  noted  here  that  in  [4,  5],  the  complete  error  e(c,  d)  in  (26)  was  not 
optimized. 

Symmetric  Spatial  Response  -  A  Special  Case  :  Many  2-D  filters  used  in  image  processing  are  symmetri¬ 
cally  shaped  in  the  spatial  domain.  Some  notable  examples  are,  Gaussian  and  Circular  filters.  In  designing  such 
spatially  symmetric  2-D  filters,  the  methods  in  [4,  5]  sometimes  produced  slightly  different  sets  of  denominator 
polynomials.  Hence,  the  estimated  spatial  response  may  not  possess  the  desired  symmetry.  This  problem  may 
be  attributed  to  separate  estimation  of  the  individual  denominators.  In  the  proposed  approach,  both  the  denom¬ 
inators  are  optimized  simultaneously  by  minimizing  the  entire  error  in  (28a).  If  necessary,  the  desired  symmetry 
may  be  imposed  by  setting,  c  =  d  in  (28a)  at  the  outset.  For  this  special  but  very  important  special  case,  (28a) 
would  have  the  following  form  : 

e(c)  A  [((I*2  -  Pc)®W*1(c))X1  +  (W*2(c)®Itl)X2]c,  (31) 

where,  the  subscripts  of  W  denote  leading  dimensions  which  may  be  unequal.  Minimization  of  the  norm  of  the 
error  in  (31)  will  result  in  a  single  set  of  optimal  coefficients  meant  for  both  dimensions.  This  is  one  of  the  major 
advantages  of  the  proposed  approach  over  the  ones  in  [4,  5]  where  separate  optimization  in  each  domain  does  not 
necessarily  guarantee  identical  denominator  coefficients  in  both  domains. 

V.  Simulation  Results 

In  order  to  demonstrate  the  effectiveness  of  the  proposed  algorithm,  the  results  of  the  design  of  a  Gaussian  Filter 
are  given  here.  The  spatial  response  of  a  quarter  plane  Gaussian  filter  defined  over  the  first  quadrant  is  given  by 

H(i,j)  =  0.256322  e[-o  io3203{(«-4)a+(j-4)2}]> 

where,  (i,j)  £  5/  and  the  support  5/  is  given  by  5/  =  {(i,j)  |  0  <  i  <  14;  0  <  j  <  14}.  The  true  or  the 
desired  spatial  response  is  shown  in  Fig.  1.  Note  that  the  spatial  response  is  symmetric  around  its  center  point. 
Fig.  3  through  5  show  the  estimated  responses  for  filter  orders  (mi  =  m2),  4,  5  and  6,  respectively.  The  results 
were  obtained  by  minimizing  the  norm  of  the  error  vector  in  (31)  with  different  orders.  The  algorithm  converged 
in  5-7  iterations.  The  plots  clearly  show  that  the  estimated  spatial  impulse  responses  match  the  desired  one 
quite  closely  and,  as  can  be  expected,  the  match  improves  as  the  filter  order  increases.  With  sixth-order  the 
difference  between  the  true  and  the  estimated  response  is  almost  negligible.  The  closeness  between  the  true  and 
the  estimated  response  was  also  measured  in  terms  of  the  ratio  of  the  power  of  the  true  response  to  that  of  the 
errors  in  each  case.  The  ratios  were  found  to  be  about  41.2dB,  61.4dB  and  86.5dB  for  filter  orders  4,  5  and  6, 
respectively.  Simulations  with  other  forms  of  2-D  filters  also  showed  similar  performance. 


127 


VI.  Conclusion  and  Future  Work  : 

An  optimal  ,2-D  HR  filter  design  method  has  been  presented.  The  algorithm  is  a  2-D  extension  of  an 
existing  optimal  1-D  approach.  The  2-D  model-fitting  criterion  has  been  decoupled  into  a  linear  and  a  nonlinear 
sub-problems.  The  non-linear  part  has  been  shown  to  possess  a  quasi-linear  relationship  with  the  unknown 
denominator  coefficients.  The  algorithm  simultaneously  optimizes  the  coefficients  in  both  dimensions.  Regarding 
future  work,  it  may  be  noted  that  similar  to  EFM  [3],  the  proposed  algorithm  is  also  applicable  for  strictly-proper 
designs  only,  albeit  in  2-D.  Recently,  an  optimal  1-D  algorithm  (OM)  which  is  applicable  for  any  general  system 
with  arbitrary  number  of  poles  and  zeros,  has  been  presented  in  [14].  Unlike  EFM,  the  general  1-D  algorithm 
in  [14]  formulates  the  criterion  entirely  differently.  It  shows  explicitly  that  the  true  error  is  linearly  related  to 
the  numerator  coefficients  whereas  the  denominator  is  nonlinearly  related.  The  possibility  of  extending  this  work 
for  designing  2-D  filters  with  any  arbitrary  numbers  of  denominator  and  numerator  orders  is  presently  under 
investigation. 
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Fig.  1  :  The  True  Spatial-Domain  response  for  a 

15  X  15  Gaussian  Filter.  ^  ^  ^st^rQate(i  Spatial-Domain  response 

with  mi  =  m  =  4 


0.30 


Hg.  3  :  The  Estimated  Spatial-Domain  response 
with  mi  =  72,-j  —  5 


Fig.  4  :  The  Estimated  Spatial-Domain 


response 
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Section  -  3.5  :  OPTIMAL  FREQUENCY  DOMAIN  DESIGN  OF  DENOMINATOR  SEPARABLE  TWO- 
Dimensional  Digital  HR  Filters 

SUMMARY 

Classical  design  techniques  using  Butterworth,  Chebyshev  or  Elliptic  polynomial  are  only  limited  particular 
types  of  design  specifications,  such  as  Bandpass,  lowpass  etc.  A  least-squares  technique  is  presented  for  designing 
quarter-plane  separable-denominator  2-D  HR  filters  to  best  approximate  prescribed  frequency  domain  (FD)  spec¬ 
ification  of  any  arbitrary  shape.  Structured  Matrix  Approximation  approach  is  utilized  to  show  that  the  FD  error 
vector  is  linearly  related  to  the  2-D  numerator  coefficients  whereas  the  relationship  with  the  2-D  denominators  is 
quasi-linear.  Furthermore,  the  numerator  and  denominator  estimation  problems  are  theoretically  decoupled.  The 
quasi-linear  relationship  with  the  denominator  is  used  to  formulate  an  algorithm  for  iterative  estimation  of  the 
denominator.  The  numerator  is  found  in  one  step  using  the  estimated  denominator.  Computer  simulations  show 
the  effectiveness  of  the  proposed  method  and  its  superior  performance  compared  to  several  existing  methods. 

I.  Introduction  : 

Design  of  2-D  digital  HR  filters  from  arbitrary  frequency  domain  specifications  is  a  highly  nonlinear  opti¬ 
mization  problem  [1-10].  Existing  designs  make  use  of  variations  of  general  nonlinear  optimization  techniques, 
such  as  Newton-Raphson  or  Fletcher- Powell  or  linear  programming  to  meet  the  prescribed  design  specifications 
[3,  4,  6-8].  But  these  general  methods  are  computationally  intensive,  highly  sensitive  to  the  choice  of  initial 
estimates  and  may  take  large  number  of  iterations.  Also,  none  of  these  methods  make  use  of  the  underlying 
matrix-structure  inherent  in  the  2-D  filter  design  problem.  For  Spatial  Domain  designs,  it  has  been  shown  by 
several  researchers  (including  the  second  author)  that  appropriate  utilization  of  the  underlying  matrix  structures 
leads  to  insightful  theoretical  framework  and  efficient  computational  algorithms  [1,  2,  5,  9,  10].  In  practice  though, 
the  filter  specifications  are  usually  prescribed  in  the  frequency-domain  and  hence,  direct  design  in  the  frequency 
domain  would  certainly  be  more  desirable.  To  the  best  knowledge  of  the  authors  no  structured  matrix  framework 
has  been  developed  for  frequency-domain  2D  HR  filter  design.  The  primary  goal  of  this  work  is  to  fill  this  gap 
by  demonstrating  that  an  equivalent  structured  matrix  framework  does  exist  in  the  frequency- domain  also  and 
furthermore,  it  can  be  utilized  equally  effectively  for  designing  2D  HR  filters.  Though  the  proposed  framework 
can  be  adapted  for  general  cases,  we  present  the  design  of  denominator-separable  filters  here  because  the  inherent 
symmetry  in  many  commonly  used  2D  filters  conform  to  the  separable-denominator  structure  and  the  stability 
of  these  filters  can  be  easily  verified. 

This  work  shows  that  the  optimal  2D  rational  model  identification  problem  belongs  to  a  special  class  of  mixed- 
nonlinear  optimization  problem  where  the  linear  and  nonlinear  parameters  appear  separately.  Furthermore,  the 
mixed  nonlinear  criterion  can  be  decoupled  into  a  -purely  linear  problem  for  estimating  the  numerator  and  a 
separate  nonlinear  problem  of  reduced  dimensionality,  for  estimating  the  separable  denominators.  The  matrix 
structure  of  the  nonlinear  denominator  criterion  naturally  leads  to  an  iterative  algorithm  whereas  the  numerator 
is  estimated  with  a  single  step  of  Least-Squares  estimation.  In  simulations,  the  proposed  approach  provides 
superior  match  than  various  existing  general  approaches. 

II.  Problem  Definition  : 

The  transfer  function  of  a  2-D  separable-denominator  LSI  system  is  given  by 


H(zi,z2) 


MzuZ2) 

B{z i)  C(z2) 


E?=oE^o«(UKS~j  =  z[Az2 

£7=0 £7=0 zibcTz2 


(1) 
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where,  b  A  [6(0)  6(1)  •  •  •i(mi)]7',  c  A  [c(0)  c(l)  •  • 


A  A 


a(0,0)  a(0, 1) 
o(l,  0)  a(l,l) 


a(0,m2)  ‘ 
a(l,m2) 


La(mi,0)  a(mi,l)  •••  a(mi,m2)J 


(2) 


and  zi  and  z2  are  vectors  of  the  form  z*  A  [1  z{  1  z{  2  •  •  •]7’  with  appropriate  sizes, 
frequency  response  be 


Xd  A 


x(wn,w2i)  z(wn,w22) 
*(w»„«2 1)  *(«!*!, W22) 


2l(wii)W2Jb3) 

I(WU„W21,) 


Let  the  Aq  x  Aq  desired 


(3) 


and  the  frequency  response  of  the  separable-denominator  filter  at  the  same  frequency  points  be  X.  Let 
Xd  A  vecp^d)  and  x  A  vec(X).  The  problem  is  to  estimate  the  coefficients  in  b,  c,  and  A  by  optimizing 
the  following  2-D  least-squares  error  criterion, 


min  ||e||2  A  ||x^  -  x||2  with  6(0)  =  l,c(0)  =  L 

b,c,A  = 


(4) 


III.  Decoupling  the  error- criterion  : 

Let  Hb(zi)  and  Hc(z2)  be  the  inverse  filters  of  B(z\)  and  C(z2)  respectively  i.e.}  B(zi)Hj)(zi)  =  1  and 
C{z2)Hc{z2)  =  1.  The  system  function  can  therefore  be  written  as 


H(zi,z2) 


A(zu_Z2) 

B(zi)C(z2) 

e?=  oE;=o«(u>^2'j 

Er=ioH*>r<  Er=vo>  2j 
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Hb(zi)  A{z\ ,  z2)  Hc(z2) 


Assuming  Aq  x  &2  significant  samples  for  the  spatial  response,  the  above  relation  can  be  expressed  in  matrix 
notation  as 


where, 


H  =  HlAHf 
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This  expression  shows  explicitly  that  the  frequency  domain  error  is  linearly  related  to  the  numerator  coefficient 
vector  a  and  nonlinear ly  related  to  the  denominators  in  a  rather  complicated  manner.  Interestingly,  if  the 
denominator  coefficients  are  known,  the  least-squares  estimate  of  the  numerator  coefficients  can  be  obtained  by 
minimizing  (4), 

a  =  (WcH£<g>W6H»)#xd  (14) 

where  #  denotes  the  pseudo-inverse  of  the  matrix.  Substituting  this  in  (10),  we  get  the  decoupled  denominator 
criterion, 

||e(b,  c)||2  =  ||xd  -  (WCH£  <g>  WbKbL)(WcHcL  <g>  W>Hi)#xd||2  =  ||(Ifcl*2  -  (PWcH|  ®  PW>h*  Ml2  (15) 

where,  Py  A  Y(YHY)~1YH  denotes  the  projection  matrix  of  a  matrix  Y  with  H  being  the  conjugate-transpose 
operator.  Extending  Theorem  2.1  in  [12],  it  can  be  shown  that  if  the  denominator  is  estimated  by  minimizing 
criterion  in  (15)  and  that  estimate  is  used  in  (14),  the  estimates  retain  the  global  optima  of  the  original  criterion 
in  (4). 

IV.  Reparametrization  of  the  error- criterion  : 

In  this  section  the  decoupled  criterion  in  (12)  will  be  directly  related  to  the  denominator  coefficients.  The 
inverse  filter  relation  B(zi)Hj)(zi)  —  1  can  be  expressed  in  matrix  notation  as 

BlH6  =  1^  (16) 
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where,  B  L  A 


6(0) 
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Let,  WhWb  =  Ifcl.  This  inverse  exists  because  the  frequencies  wu’s  are  distinct.  Using  it  in  (16)  with  (17)  and 

(18), 
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The  bottom- left  corner  element  of  the  matrix  at  right  suggests  that  W^B  and  WH]  are  orthogonal,  i.e., 
(W«B)ff(WH‘)  =  0.  Also,  since  rank( W^B)  +  rank(WHbL)  =  (ki  -  ni  —  1)  +  («i  +  1)  =  fci,  using  a  theorem 
on  projection  matrices, 

PwJ|B  +  PwHj  =^r  (20) 

Similarly,  from  the  inverse  filter  relation  C(z2)Hc(z2)  =  1,  we  can  get 


P\v"c  +  Pwhcl  =  Ijb2-  (21) 

CJ  L 

Substituting  the  above  relations  in  (12)  and  using  Kronecker  product  representation  (<g>),  the  error  can  be  written 
as 

e(b,c)  {zt.  1(1^2  "  Pwc^  c)  ®  P  wj^  b  +  Pwjj  b  ®  til  xd  =  [((Ifc2  —  Pw^c)  (S)Vj)X1  (Vc<g>Ifcl)X2]  ^  c  ^  *  where, 

(22) 

X1  and  X2  are  formed  with  prescribed  data,  V&  A  (W^B)  ( (W^B)^  (Wf[B)  )-1  and 
Vc  A  (Wc^C)((W^C)^(W^C))-1. 


V.  Simulation  Results  : 

Several  designs  were  implemented  using  the  proposed  algorithm  and  the  performances  were  compared  with 
existing  approaches.  Fig.  1-3  show  the  results  of  the  comparison.  Fig.  la,  2a  and  3a  show  results  using  the 
methods  proposed  in  [6],  [7]  and  [8],  respectively.  For  the  same  or  less  numerator /denominator  orders,  Fig.  lb,  2b 
and  3b  show  the  corresponding  results  using  the  proposed  method.  The  relative  rms  errors  [7]  for  the  results  in 
Fig.  la,  2a  and  3a  are  0.67,  0.28  and  0.77,  respectively.  The  errors  for  the  results  in  Fig.  lb,  2b  and  3b  are  0.21, 
0.26  and  0.68  respectively.  Clearly,  the  proposed  approach  found  better  match  with  lesser  number  of  coefficients, 
in  all  cases.  The  number  of  iterations  for  the  proposed  approach  were  less  than  10  in  all  cases,  whereas  the  general 
optimization  approaches  sometimes  took  close  to  hundred  or  more  iterations. 
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Section  -  3.6  :  OPTIMAL  SPATIAL-DOMAIN  DESIGN  OF  2-D  HR  FILTERS 


Summary 

In  this  Section  we  present  a  structured  matrix  approximation  framework  to  develop  the  most  general  form 
for  optimal  least-squares  (LS)  design  of  2-D  recursive  filters  from  prescribed  spatial  domain  data.  Unlike  the 
work  in  Section  3.4,  no  separability  is  assumed  for  the  2D  denominator.  Utilizing  matrix  structures  inherent 
in  this  problem  it  is  shown  that  the  exact  £2  error  has  a  purely  linear  relationship  with  the  2-D  numerator 
parameters  whereas  the  2-D  denominator  coefficients  are  nonlinearly  related  to  the  error.  But  more  interestingly, 
the  denominator  and  numerator  estimation  problems  are  theoretically  decoupled  into  separate  problems  without 
affecting  any  optimality  properties.  In  the  decoupled  form,  the  numerator  estimation  problem  is  shown  to  be 
purely  linear.  For  estimating  the  denominator  also,  it  is  shown  that  the  decoupled  £2  error  vector  possesses  a 
quasi-linear  relationship  with  the  denominator  coefficients.  Decoupled  estimation  leads  to  reduced  computational 
complexity  because  there  is  no  need  for  iterating  on  the  numerators.  Simulation  results  indicate  that  for  several 
common  filer  design  problems,  the  proposed  general  version  performs  better  than  the  separable  design  developed 
earlier  in  Section  3.4. 

Introduction 

Many  2-D  filter  synthesis  algorithms  have  been  developed  by  extending  existing  algorithms  for  1-D  filter 
design.  Specifically,  Shanks  et  al  [1]  extended  the  work  of  Shanks  [5];  Cadzow  [2]  and  Shaw  and  Mersereau  [3] 
utilized  many  of  the  general  non-linear  optimization  methods;  and  Shaw  and  Mersereau  [3]  also  extended  the  work 
of  Steiglitz  and  McBride  [6].  But  these  methods  are  suboptimal  in  the  sense  that  they  do  not  optimize  the  exact 
fitting  error  criterion.  In  contrast  to  these  approaches,  the  iterative  method  (EFM)  proposed  by  Evans  and  Fischl 
[7]  is  optimal  in  the  1-D  case  because  it  does  optimize  the  true  error  criterion.  There  have  been  some  previous 
attempts  in  generalizing  EFM  to  2-D  also  [10-13]  but,  as  shown  in  this  paper,  the  complete  error  criterion  for 
the  most  general  case  has  not  yet  been  developed  or  optimized.  Even  the  suboptimal  error  criterion  had  not  not 
optimized  w.r.t .  the  filter  coefficients  in  two  dimensions  simultaneously.  The  Evans-Fischl  method  has  been  found 
to  be  highly  accurate  for  1-D  filter  design.  In  a  recent  work,  we  have  extended  1-D  EFM  to  designing  2-D  HR 
filters  with  separable  denominators  [8],  where,  unlike  several  existing  2-D  methods  [1,3,10-13],  the  exact  fitting 
error  was  minimized  w.r.t.  the  filter  coefficients  in  both  dimensions  simultaneously.  Simultaneous  optimization 
was  shown  to  be  effective  for  some  commonly  occurring  design  problems  with  symmetric  spatial  response. 

In  this  paper  we  present  a  structured  matrix  approximation  framework  to  develop  the  most  general  2-D 
version  of  EFM  for  optimal  least-squares  (LS)  design  of  2-D  recursive  filters  from  prescribed  spatial  domain 
data.  Utilizing  matrix  structures  inherent  in  this  problem  it  is  shown  that  the  exact  £2  error  has  a  purely  linear 
relationship  with  the  2-D  numerator  parameters  whereas  the  2-D  denominator  coefficients  are  nonlinearly  related 
to  the  error.  But  more  interestingly,  these  two  sets  of  parameters  appear  separately  in  the  2-D  LS  criterion. 
Hence,  using  a  theorem  on  separability  from  Numerical  Analysis  literature,  it  is  shown  that  the  numerator  and 
denominator  estimation  problems  can  be  mathematically  decoupled  without  affecting  any  optimality  properties. 
In  the  decoupled  form,  the  numerator  estimation  problem  is  shown  to  be  purely  linear.  For  estimating  the 
denominator  also,  it  is  shown  that  the  decoupled  £2  error  vector  possesses  a  quasi-linear  relationship  with  the 
denominator  coefficients.  Decoupled  estimation  leads  to  reduced  computational  complexity  because  there  is  no 
need  for  iterating  on  the  numerators.  Simulation  results  indicate  that  for  several  common  filer  design  problems,  the 
proposed  general  version  performs  better  than  the  separable  design  developed  earlier  by  the  Principal  Investigator. 
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Formulation  of  the  Problem 


Let  the  prescribed  first-quadrant  spatial  impulse  response  of  size  k\  x  k2  be  given  by 


X 


*(0,0) 

*(1,0) 


*(0,1) 

*(1.1) 


*(0,&2  -  1) 

x(l,k2  -  1) 


Let 


L  —  1,0)  x(ki-lyl)  •••  x(k\  —  1,  k2  -  1)  J 
H(zi  z2)  ~  A{zuz*)  _  oa(i)j)zi  %zi* 


(i) 


(2) 


ITJo  Ejm=0  Kh  j)zIiz2j 

be  the  transfer  function  of  the  2-D  filter  to  be  designed  to  approximate  X  and  let  its  ki  x  k2  spatial  response  be 


given  by 


H  = 


h(0, 0) 

ft(i.o) 


K  0,  i) 
K  1,1) 


h(0,k2-l) 
h(l,k2  —  1) 


(3) 


.h(kx  -1,0)  A(jfci  —  1, 1)  •  ••  h(ki  -  1,  Jb2  —  1)J 

In  order  to  develop  the  structured-matrix  representation  of  the  2-D  filter  design  problem,  define  two  matrices 
containing  the  numerator  and  denominator  coefficients  as 


A  = 


«(0,0) 

®(o, 1)  •• 

■  a(0,ri2)  ‘ 

«(1,0) 

o(l, 1)  •• 

•  a(l,«2) 

G  jp#(ni+1)x(na+1) 

_a(ni,0) 

a(ni,l)  •• 

•  a(ni,n2). 

f  6(0,0) 

6(0,1) 

6(0,  rm 

*)  1 

(4) 


B  = 


K  1.0)  6(1,1) 


6(1,  m2) 


G  ]R(mi+1)><(m2+1) 


(5) 


.6(mi,0)  6(m1}l)  ••• 

respectively.  In  vector  form  define,  x  =  ?;ec(X),  h  =  uec(H),  a  =  vec(A)  and  b  —  vec(B)  where  vec  is  the 
operation  of  stacking  all  the  columns  of  a  matrix  one  below  the  other.  The  problem  addressed  in  this  paper  is  to 
estimate  a  and  b  by  minimizing  the  following  -£2-norm  of  the  error  between  x  and  h  i.e., 


mm  ||  e  | 

a,b 


-h 


(6) 


Decoupling  the  Error  Criterion 

Equation  (2)  can  be  rewritten  as 


A(zi,z2)  =  H(zi,z2)B(zi,z2). 


(7) 


Note  that  k\  x  k2  significant  samples  of  the  desired  spatial  response  are  to  be  matched.  Hence,  by  equating  on 
both  sides  of  (7)  the  coefficients  of  equal  powers  of  zf1  and  z^1  up  to  k\  —  1  and  k2  —  1,  respectively,  equation 
(7)  can  be  expressed  using  matrix-vector  notation  as 


a 

H1' 

B1' 

= 

b  = 

0 

El2 

Bt 
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where  H1,  H2,  B1  and  BT  are  defined  in  the  Appendix.  If  the  impulse  response  H  and  the  numerator  coefficients 
b  were  known,  the  numerator  coefficients  a  can  be  calculated  from  the  top  part  of  (8)  as 


a  =  H1  b  =  B1  h.  (9) 

However,  in  this  case  the  exact  H  is  unknown.  Hence,  replacing  H2  in  the  lower  half  of  (8)  by  X2  formed  using 
the  corresponding  elements  of  X,  would  produce  the  following  ‘equation  error’, 

d(b)  A  X2b  =  BTx.  (10) 

From  (6),  x  =  h  +  e.  Using  this  in  (10),  we  get 

d(b)  =Bt  (h  +  e) 

=BTe,  using  (8)  (11) 

Also  according  to  (8),  B  is  orthogonal  to  h.  Hence,  based  on  the  orthogonality  principle  [9]  an  inverse  relationship 
can  be  established  using  similar  steps  as  in  [7,  8], 


e  =B(BtB)-1Btx 
A  WBtx 
=WX2b 


A  W 


b 


=Wg  +  WGb 


(12) 


where  b  A  [1  :  bT]T ,  g  is  the  first  column  of  X2  and  G  contains  the  remaining  columns  of  X2.  If  W  is  assumed 
to  be  independent  of  b,  by  setting  the  gradient  of  ||e||2  in  (13)  to  zero,  the  denominator  vector  b  can  be  estimated 


as, 


b  =  —  (GTWTWG)-1GTWTWg 


(13) 


But  since  W  does  depend  on  b,  the  above  equation  will  be  used  to  estimate  bW  iteratively,  with  W  formed  using 
b  !~  V>  estimated  at  the  previous  iteration.  A  convenient  initial  estimate  of  b  can  be  obtained  by  minimizing  the 
equation  error  in  (10)  as 


fc(°)  = 


1 

-(GtG)_1Gt 


(13) 


Simulation  Results 

Computer  simulations  have  been  done  to  design  zero-phase  lowpass  [14] ,  zero-phase  bandpass  [14] ,  Gaussian 
and  Laplacian  filters.  It  has  been  found  that  for  same  filter  orders,  the  proposed  method  performs  better  than  the 
separable-denominator  case  [8]  for  the  lowpass  and  the  bandpass  cases,  while  for  the  Gaussian  and  the  Laplacian 
cases  the  separable-denominator  method  [8]  appears  to  perform  better.  Also,  in  all  the  examples,  the  proposed 
method  performed  better  than  the  modified  Prony’s  method  given  in  [14],  Some  plots  have  been  included  of  the 
obtained  designs. 


Appendix 
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Definitions  of  the  matrices  used  in  (8)  are  as  follows.  It  should  be  noted  that  the  matrix  structures  have 
been  given  only  for,  «i  +  1  <  mi,  n2  +  1  <  mi,  the  modifications  for  other  cases  being  obvious. 
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bt  a 
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Separable-Denominator  case 


Optimal  General  Case 
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Section  -  3.7  DISTRIBUTED  LOOK-AHEAD  :  A  GENERAL  APPROACH  FOR  PIPELINING  RECURSIVE 

Digital  Filters 
Summary 

A  new  Look-Ahead  (LA)  scheme,  Distributed  Look-Ahead  (DLA),  is  proposed  for  pipelined  implementation 
of  recursive  digital  filters.  It  is  established  that  in  case  of  many  recursive  filters,  DLA  can  provide  equivalent 
and  stable  implementation  with  reduced  pipeline  delay  and  hardware  complexity,  when  compared  with  some 
existing  LA  schemes  [4,  5].  The  existing  Scattered  Look-ahead  implementation  [5]  achieves  stability  at  the  cost 
of  increased  multiplication  and  latch  complexities  and  considerable  delay  in  output  generation.  The  Clustered 
look-ahead  approach  can  not  always  guarantee  stability  [5].  This  work  shows  that,  in  order  to  attain  stability,  the 
output  samples  need  not  be  clustered  or  equally  scattered.  Indeed,  in  many  filter  design  problems,  stability  can 
be  maintained  by  using  unequally  distributed  past  output  samples.  When  compared  with  the  scattered  approach, 
the  proposed  scheme  uses  fewer  number  of  pole-zero  cancelations  and  the  introduced  roots  are  not  necessarily  at 
the  same  radii  as  the  original  filter  poles.  Hence,  the  proposed  DLA  scheme  has  reduced  multiplication  and  latch 
complexities,  higher  area-efficiency  and  it  produces  outputs  with  reduced  delays. 

1.  Introduction 

Look-ahead  pipelining  is  highly  effective  in  attaining  high  sampling  rate  and  computation  speed  for  low- 
cost  VLSI  implementation  of  digital  HR  filters  [3-5,  9,  13].  Among  existing  LA  schemes,  the  Clustered  (CL A)  or 
Time-Domain  (TD)  approach  [3,  4,  9]  generates  the  present  output  using  contiguous  past  output  samples  whereas 
the  Scattered  (SLA)  or  2-domain  (ZD)  approach  [5,  12]  uses  equally  separated  past  output  samples.  A  desirable 
feature  of  SLA  is  that  stability  is  guaranteed.  However,  this  is  achieved  at  the  cost  of  relatively  large  delay  in 
output  generation  as  well  as  increased  multiplication  and  latch  complexities  to  implement  the  numerator.  On 
the  other  hand,  CLA  may  require  filter  order  augmentation  to  maintain  stability,  which  also  comes  at  the  cost  of 
increased  hardware  complexity  [3]. 

In  this  paper,  we  propose  a  general  Look-Ahead  approach  that  opens  up  a  large  class  of  new  possibilities  to 
provide  stable  realizations  with  reduced  pipeline  delay  and  hardware  complexity  than  the  existing  LA  schemes. 
Possible  stability  regions  for  the  proposed  scheme  are  also  addressed.  This  paper  argues  that,  in  order  to  attain 
stability  in  pipelined  form,  the  output  samples  need  not  be  clustered  [3,  4,  9,  13]  or  equally  scattered  [5,  12]. 
Indeed,  for  many  filter  design  problems,  it  is  shown  that  stability  can  be  maintained  by  using  unequally  distributed 
past  output  samples.  The  proposed  scheme  is  denoted  as  Distributed  Look-Ahead  (DLA)  approach,  where  the 
number  of  denominator  coefficients  can  be  kept  same  as  that  in  the  original  filter  (as  in  SLA)  but  the  numerator  and 
denominator  orders  are  lower  than  that  in  SLA.  Hence,  the  proposed  DLA  scheme  can  offer  reduced  multiplication 
and  latch  complexities,  higher  area-efficiency  and  it  can  produce  outputs  with  reduced  pipeline  delay  than  the  SLA 
scheme.  Unlike  SLA  but  similar  to  CLA,  stability  is  not  guaranteed  with  the  proposed  scheme.  However,  simple 
stability  conditions  and  regions  can  be  derived  and  have  been  presented  for  various  pipeline  stages.  It  is  also 
demonstrated  that  the  numerator  and  denominator  polynomials  can  be  factorized  into  lower-order  polynomials, 
which  can  further  simplify  hardware  implementation  and  complexity.  Examples  are  included  to  demonstrate  the 
validity  of  the  proposed  approach. 

The  paper  is  organized  as  follows.  In  Section  2,  two  existing  LA  schemes  are  briefly  summarized.  In  Section 
3,  the  general  Look-Ahead  scheme  is  proposed  along  with  some  examples.  In  Section  4,  Stability  conditions  are 
presented  for  pipelining  of  a  2nd-order  recursive  filters  using  the  proposed  DLA  scheme  and  in  Section  5,  several 
examples  are  provided  to  demonstrate  that  stable  pipelined  implementations  with  DLA  can  be  achieved  with 
reduced  hardware  complexity. 
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2.  Existing  Look-Ahead  Schemes 

Consider  a  recursive  digital  filter  with  L-th  order  numerator  and  N-th  order  Denominator  of  the  form, 


H(z)  = 


ELA-*-'  _  B(z) 


(1) 


which  is  to  be  implemented  in  pipelined  form.  The  two  major  existing  Look-Ahead  forms  are  briefly  outlined 
first. 


2.1.  Clustered  Look-Ahead  (CL  A)  :  M-stage  CL  A  pipelining  of  the  filter  in  (1)  would  have  the  following 
form  [3,  4,  9,  13]  : 

=  1  +  bj£±  +  62z~2  +  •  •  •  +  m 

1  +  O-M Z~M  +  aM+ lZ“(M+1)  + - h  aM+N-lz~^M+N~^ 

Note  that  the  denominator  coefficients  are  ordered  in  a  clustered  form.  The  numerator  (non-recursive  portion) 
can  be  implemented  by  N  +  M  multiplications  and  the  denominator  (recursive  portion)  can  be  implemented  with 
N  multiplications  .  Thus,  the  total  multiplication  complexity  is  (2 N  +  M)  and  the  latch  complexity  is  linear  in 
M.  The  extra  delay  in  producing  output  is  M . 


2.2.  Scattered  Look-Ahead  (SLA)  :  An  equivalent  M-stage  pipelining  of  the  same  iV-th  order  recursive  filter 
can  be  obtained  by  [5,  12], 


Hm 


(z)  = 


1  +  61Z-1  +  62Z~2  +  •  •  •  +  bN(M-i  )+£2r-(JV(M-1)+£) 
1  +  d\[Z~M  +  0,2Mz~2M  H - 1-  dNMz~NM 


(3) 


Note  that  the  non-zero  denominator  coefficients  are  equally  ’scattered’.  The  multiplication  complexity  for  the 
non-recursive  portion  of  the  pipelined  implementation  is  ( NM  +  1)  and  that  of  the  recursive  portion  is  N.  Thus, 
the  total  multiplication  complexity  is  (NM  +  M  +  1)  and  the  latch  complexity  is  square  in  M.  The  extra  delay 
in  producing  output  is  (N M  —  N)y  which  may  be  significant  because  of  high  order  of  the  filter  in  pipelined 
form.  However,  if  M  is  a  power  of  2,  then  using  a  decomposition  technique  the  total  multiplication  and  latch 
complexities  can  be  further  reduced  [5]. 

3.  The  Proposed  Distributed  Look-Ahead  Pipelining  (DLA) 

In  this  new  look-ahead  scheme,  the  filter  transfer  function  is  transformed  to  have  the  form, 


H%(z)  = 


1  +  61Z-1  +  •  •  •  +  bM+kL-N+LZ-(M+kL~N+L) 


l  +  dMZ  M+dM-\-k1Z  +  dM+k2z  - \-dM+kLz  + 


(4) 


where,  in  general,  can  be  arbitrary  integer  values  with  =  A,  in  order  to  keep  the  total  number 

of  denominator  (at)  coefficients  same  as  in  the  original  denominator  in  (1).  It  is  easy  to  show  that  the  two 
existing  look-ahead  schemes  defined  in  equations  (2)  and  (3)  are  special  cases  of  this  general  M-stage  look-ahead 
representation.  Specifically,  in  case  of  CLA,  ki  =  i  and  for  SLA,  k{  =  Mi ,  with  k^  =  (N  —  1  )M.  Clearly,  if  the 
denominator  order  of  DLA,  (M  +  k^)  is  less  than  the  order  for  SLA  (AM),  the  proposed  LA  scheme  would  offer 
considerable  hardware  savings  over  SLA. 

A  comparison  of  the  multiplication  complexities  of  the  three  pipelining  schemes  are  given  in  Table  1.  Note  that  if 
M  is  a  power  of  2,  then  using  a  decomposition  technique,  the  total  multiplication  complexity  of  the  SLA  scheme 
can  be  further  reduced  to  L  +  N  +  N  log 2(M). 


4.  DLA  Based  Pipelining  of  Second- Order  Filter  Blocks 
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In  this  section,  an  iterative  scheme  is  given  first  for  determining  the  coefficients  for  pipelining  second-order 
(N  =  2)  filter  blocks.  Then  several  examples  of  DLA  implementation  are  given  for  different  values  of  M. 


4.1.  Iterative  Scheme  for  Obtaining  Augmentation  Polynomial  :  Consider  a  second  order  filter  transfer 
function, 


H(  v  =B(z)  Bjz) 

y>  A(z)  1  +  aiZ"1  +a22"2 


(5) 


For  DLA-based  pipelining  of  this  second-order  filter,  the  only  choice  is,  k\  =  Icl  =  2  in  (4).  To  determine  the 
DLA  pipelined  coefficients  from  these  serial  coefficients,  it  may  be  noted  that  Hp  (z)  must  equal  the  original 
H(z)  and  hence,  Hp  (z)  can  be  obtained  by  multiplying  an  augmentation  polynomial  D(z)  in  the  numerator  as 
well  as  the  denominator  of  (4),  i.e, 


H%(z) 


B{z)D{z) 

A{z)D{z) 


H(z) 


(6) 


where,  the  coefficients  of  D(z)  =  1  +  d\z~l  +  d^z~2  + - \-  (Imz~M  should  be  selected  such  that  the  denominator 

possesses  the  desired  DLA  form  in  (4).  It  can  be  shown  that  the  coefficients  of  the  D(z)  polynomial  can  be  found 
recursively  using  the  following  steps, 

Initialize  :  do  =  1,  d\  =  —a i  and 

Iterate  :  for  i  =  2  to  M  —  1 


d{  =  -~ot\di-.\  —  0Lid\-2 


(7) 


4.2.  Examples  :  Let  the  complex  conjugate  poles  of  the  second  order  filter  block  be  located  at  z  —  re^6 .  Then 
the  transfer  function  of  the  second  order  filter  would  have  the  form, 


1 


^  ^  1  —  2r  cos  0Z”"1  +  r2z“2 


(8) 


where,  the  numerator  has  been  set  to  unity  without  any  loss  of  generality.  Using  ai  =  2rcos0  and  c*2  =  r2  in 
the  iterations  of  (7),  a  4-stage  (M=4)  DLA  pipelined  filter  can  be  shown  to  have  the  form, 


4  l  +  2rcos0z  1  4-  r2(2cos20  +  l)z  2  -f  4r3  cos  6  cos  26 z  3  +  2r4  cos  26z  4 
D  Z  1  —  r4(2  cos  46  +  l)z-4  +  2r6  cos  20z-6 

Interestingly,  this  transfer  function  can  be  further  factorized  into  a  more  convenient  decomposed  form  as, 

(1  +  2r  cos  6z~l  -f  r2z-2)(  1  +  2r2  cos  20z“2) 


i  (  »\  - 


HUz) 


(1  —  2r  cos  6z~l  +  r2z~2)(  1  4-  2rcos  6z~l  +  r2z~2)(  1  +  2r2  cos  20z-2) 


(9) 


(10) 


Implementation  of  this  decomposed  form  allows  hardware  savings  over  its  SLA  counterpart.  Using  similar  steps, 
it  can  be  further  shown  that  3,  6  and  8-stage  DLA  implementations  of  the  second-order  filter  have  the  following 
decomposed  forms,  respectively, 


H3d(z)  = 


1  +  2r  cos  Oz  1  -f  r2(2  cos  26  +  l)z  2  -f  (2  cos  26  +  1  )z 
-q{2  cos  40  +  2  cos  26  +  5)z-3  +  2 cos e  cos  ^  +  ^)z~ 


-3 


1- 


2  cos 


H%{z)  = 


(l  +  2rcos0z  1  +  r2z  2)(1  +  2r2  cos  26z  2  +  r4(4  cos2  26  —  l)z  4) 

(1  —  2 r  cos  6z~ 1  +  r2z“2)(  1  +  2 r  cos  6z~l  +  r2z"2)(  1  +  2r2  cos  20z~2  +  r4(4  cos2  26  —  l)z“4) 


(11) 

(12) 
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(1  +  2r  cosOz  l+r2z  2)(1  +  2r2  cos  9z  2  +  r4(2  cos  40  +  l)z  4  +  r6(4  cos  40  cos  20  +  l)z-6) 

D  (1  -  2 rcos6z~‘l  +  r2z_2)(  1  +  2r  cos0z_1  +  r2z_2)(l  +  2r2  cos  2 6z~2  +  r4(2  cos  40  +  l)z“4  +  r6(4cos40cos  20  +  . 

(13) 

A  comparison  of  the  hardware  complexities  between  the  SLA  and  the  DLA  schemes  for  M  =  3,4,6  and  8  after 
their  respective  decompositions  is  given  in  Table  2. 

From  this  table  it  is  apparent  that  for  all  stages  of  pipelining  the  DLA  scheme  has  a  definite  edge  over  the 
SLA  scheme  as  far  as  hardware  complexity  is  concerned.  Next,  the  stability  conditions  for  the  DLA  scheme  are 
established  for  second-order  filter  blocks  for  a  several  values  of  M . 


4.3.  Stability  conditions  :  Consider  the  general  second  order  filter  block  with  complex  conjugate  poles  at 
z  =  represented  by  the  transfer  function  in  equation  (8). 

4.3.1.  M  =  4  case  :  The  4-stage  DLA  pipelined  implementation  of  the  second  order  filter  is  obtained  by  using 
the  general  iterative  scheme  discussed  above.  The  transfer  function  of  this  is  as  in  equation  (10). 

This  4-stage  pipelined  implementation  will  be  stable  if  the  roots  of  the  factor,  (1  +  2r2cos20z“2)  are  less  than 
unity.  It  can  be  shown  that  this  would  be  true  if  0  <  0.5  cos-1  (^p) .  The  region  satisfying  this  stability  condition 
is  shown  in  Fig.  1  as  the  shaded  area. 

Hence,  if  for  any  4-stage  pipelined  implementation  of  CLA  produces  an  unstable  filter,  but  it  is  found  that  the 
condition  on  0  stated  above  or  in  Fig.  1  is  satisfied,  then  using  the  proposed  DLA  transformation  would  definitely 
be  more  appropriate  than  the  SLA  in  (3),  because  the  later  would  require  extra  hardware  for  implementing  both 
the  numerator  and  the  denominator.  The  exact  savings  in  hardware  for  this  case  can  be  found  in  Table  2. 

4.3.2  M  =  6  case  :  Consider  the  6-stage  implementation  of  the  2nd-order  filter  which  has  the  convenient  factored 
form  shown  in  equation  (12). 

Because  of  the  convenient  quadratic  factored  form,  the  stability  region  is  easy  derive  and  is  displayed  in  Fig.  2. 
It  is  interesting  to  note  that  these  decompositions  have  simple  power  of  2  factors  and  hence,  the  corresponding 
hardware  complexities  are  less  than  the  SLA  decomposition  scheme  given  in  [5].  The  hardware  savings  being  2 
multiplier- adder  units  and  8  latches  (refer  to  Table  2). 

Similar  stability  regions  can  be  evaluated  for  other  M  values. 

5.  EXAMPLES 


5.1.  Example  with  M  —  4  :  It  had  been  shown  in  [5]  that  the  second  order  transfer  function 

=  1  —  5/4*-1  +  3/82-2 


(14) 


produces  unstable  filter  with  CLA  implementation  (refer  to  Figure  3(a)  ).  Using  the  DLA  formulae  presented 
above,  it  can  be  shown  that 


the  4-stage  implementation  is  given  as, 


rr4  ,  x  (1  +  1.25z_1  +  0.3750z2)(l  +  0.8125z"2) 
= - l-0.1K».-  +  0.1143.-« - 


Using  the  SLA  technique, 


4  _  (1  +  1.25*-1  +  0.3750*9)(1  + 0.8125*"* +  0.1406*"4) 

l-0.3789z-4  +  0.0198z-8 


(15) 


(16) 
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Pole-zero  plots  of  the  4-stage  DLA  and  SLA  implementations  are  shown  in  Figures  3(b)  and  3(c).  The  DLA  and 
the  SLA  implementations  for  this  example  is  shown  in  Figures  4  and  5  respectively.  They  clearly  demonstrate  the 
hardware  savings  for  the  DLA  case.  Table-2  also  shows  the  gain  in  hardware  requirements  with  the  DLA  case. 


5.2.  Example  with  M  —  6  :  Consider  the  second  order  transfer  function 


v  7  l-lAz-l  +  0.bz~2'  v  ' 

A  6-stage  CL  A  implementation  of  equation  (17)  yields  an  unstable  pipelined  filter.  The  pole-zero  plot  of  the 
unstable  CLA  filter  is  shown  in  Figure  6(a).  Using  the  DLA  formulae  given  above,  it  is  shown  that' the  6-stage 
DLA  implementation  is  stable  (refer  to  Figure  6(b))  and  is  given  as, 


H%{z)  = 


(1  +  1.4z-1  +  0.5z~2)(l  +  0.96z-2  +  0.6716z"4) 


l-0.4132z-6  +  0.1825z- 


and  the  corresponding  SLA  implementation  is 


5  (1  +  1.4z_1  +  0.5z~2)(l  +  0.96z-2  +  0.6716z-4  +  0.2400z-6  +  0.0625z-8) 

1  —  0.1647z-6  4-  0.0156z-12 


The  pole-zero  plot  for  the  SLA  implementation  is  shown  in  Figure  6(c)  and  the  hardware  savings  for  the  DLA 
case  over  the  SLA  case  for  this  example  is  also  given  in  Table- 2. 

Hence,  in  all  these  cases,  there  is  no  need  to  use  SLA  for  stable  pipelined  realizations  and  considerable 
hardware  savings  and  reduced  pipelining  delays  can  be  achieved  if  DLA  is  used  instead. 
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Pipelining 

Methods 

Multiplication 

Complexity 

Delay 

in  producing 
First  output 

CLA 

L  +  M  +  N-l 

M 

SLA 

NM  +  L 

NM 

DLA 

M  +  kL  -  N  +  2L  +  1 

M  +  kL 

Table  1:  Comparison  of  Hardware  Complexities  between  the  Various  Pipelining  Techniques 


Pipeline 

Stages 

Pipeline 

Method 

Number  of 
Multiplier 
/Adder 
Units 

Number 

of 

Latches 

Delay 

in 

producing 

First 

Output 

M  =  3 

SLA 

6 

10 

6 

DLA 

5 

8 

5 

M  =  4 

SLA 

6 

14  ^ 

8 

DLA 

5 

10 

6 

M  =  6 

SLA 

8 

22 

12 

DLA 

6 

14 

8 

M  =  8 

SLA 

8 

30 

16 

DLA 

7 

18 

10 

Table  2:  Comparison  of  Hardware  Complexities  for  DLA  Pipelining  for  various  M 


theta  in  radians 


Figure  1:  Stability  Regions  for  M  =  4  Case  Figure  2:  Stability  Regions  for  M  —  6  Case 
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Section  -  3.8  OPTIMAL  LEAST-SQUARES  DESIGN  OF  PIPELINED  RECURSIVE  FILTERS  IN  THE  TlME- 
DOMAIN 

Summary 

Currently,  look-ahead  (LA)  pipelined  recursive  filters  are  obtained  primarily  via  transformation  of  a  given 
un-pipelined  transfer  function  [3-5,  9,  13].  For  these  approaches,  it  is  assumed  that  the  un-pipelined  transfer 
function  has  already  been  designed  as  an  intermediate  step.  In  this  Section,  we  present  a  new  algorithm  (0M- 
LA)  for  direct  and  optimal  estimation  of  the  coefficients  of  recursive  filters  in  look-ahead  pipelined  form.  OM-LA 
is  developed  by  appropriate  modification  of  a  recently  proposed  optimal  method  (OM)  for  designing  un-pipelined 
filters  [7].  It  is  demonstrated  that  the  proposed  one-step  approximation  can  achieve  superior  match  with  reduced 
pipelined  filter  order  because  it  does  not  rely  on  pole-zero  cancelations  as  in  current  LA  pipelining  approaches. 
It  is  also  shown  that  the  denominator  polynomial  can  be  constrained  to  possess  any  of  the  possible  look-ahead 
configurations.  Unlike  some  existing  methods  [1-3],  OM-LA  minimizes  the  true  time-domain  fitting  error-norm 
between  the  prescribed  and  the  estimated  impulse  response  and  produces  superior  results.  Several  examples  are 
provided  to  illustrate  the  effectiveness  of  the  proposed  approximation  algorithm. 

1.  Introduction  Look-ahead  pipelining  is  highly  effective  in  attaining  high  sampling  rate  and  computation 
speed  for  low-cost  VLSI  implementation  of  digital  HR  filters  [3-5,  9,  13].  It  may  be  noted  that  the  original  LA 
schemes  for  pipelining  recursive  filters  [3-5]  (including  DLA  proposed  in  the  previous  Section),  consist  of  two 
steps.  First,  an  un-pipelined  (or  ‘Serial’)  filter  is  assumed  to  be  available,  i.e.,  the  transfer  function  of  the  filter  in 
serial  form  is  assumed  to  have  been  approximated  by  matching  some  prescribed  or  desired  specification.  The  LA 
transformations  are  then  introduced  as  a  second  step  when  the  filter  coefficients  in  pipelined  form  are  obtained  by 
applying  either  CLA,  SLA  or  DLA  transformation  on  the  serial  filter  coefficients.  The  LA  schemes  differ  in  the 
way  order  augmentation  of  numerator  and  denominator  polynomials  is  achieved.  Mathematically,  the  inherent 
transfer  function  remains  exactly  identical  before  and  after  any  LA  transformation  is  applied.  The  higher  orders 
in  the  pipelined  cases  are  accounted  for  by  pole-zero  cancelation  which  has  no  effect  on  the  filter’s  response  or  its 
transfer  function.  In  this  part  of  the  project,  a  direct  approach  is  proposed  for  approximating  Recursive  filters 
having  desired  Look-Ahead  pipelined  forms. 

A  significant  drawback  of  the  current  two-step  approach  to  pipeline  recursive  filters  is  that  the  degrees  of 
freedom  offered  by  the  higher  orders  in  the  pipelined  filters  are  not  exploited  in  any  way.  Moreover,  for  finite 
precision  implementation  using  limited  number  of  bits,  pole-zero  cancelation  may  cause  numerical  implementation 
problems.  Hence,  a  key  motivation  for  the  later  part  of  the  paper  is  to  explore  if  the  look-ahead  recursive  filters 
are  designed  directly  in  a  single  step,  superior  approximation  can  be  achieved  at  lower  pipelined  filter  order  while 
avoiding  the  pole-zero  cancelation  problems  associated  with  the  current  two-step  design  process.  In  these  regards, 
it  may  noted  that  frequency  domain  approaches  have  been  considered  in  [1]  while  a  time-domain  approach  had 
been  taken  in  [2],  though  only  the  modified  least-squares  error  criterion  has  been  minimized.  In  this  paper,  a 
general  theoretical  framework  for  direct  and  optimal  Least-Squares  estimation  of  coefficients  of  pipelined  digital 
HR  filters  in  the  time  domain  is  presented.  The  proposed  approximation  approach  is  developed  by  appropriate 
modification  of  a  recent  work  by  the  first  author  on  optimal  time-domain  approximation  of  recursive  digital 
filters  [7].  The  true  nonlinear  error  criterion  is  theoretically  decoupled  into  two  separate  sub-problems  of  lower 
computational  complexities.  Estimation  of  the  numerator  is  a  linear  single-step  problem  whereas  the  non-linear 
denominator  criterion  possesses  a  weighted  quadratic  form  that  is  convenient  for  iterative  optimization.  It  is 
shown  with  several  examples  that  the  proposed  approach  can  produce  pipelined  filters  with  better  match  to 
prescribed  specs  with  much  lower  filter  orders  and  without  any  pole-zero  cancelations. 
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The  paper  is  organized  as  follows.  In  Section  2,  existing  LA  schemes  are  briefly  summarized.  In  Section  3, 
the  one-step  Look-Ahead  pipelined  filter  approximation  method  is  presented  and  in  Section  4,  some  simulation 
examples  are  provided  to  illustrate  the  effectiveness  of  the  direct  approximation  approach. 

2.  Transfer  Functions  for  various  Look-Ahead  Schemes 

Consider  a  recursive  digital  filter  with  L-th  order  numerator  and  N- th  order  Denominator  of  the  form, 


H(z) 


E  Uhl 


B{z) 


(i) 


1-EL 

which  is  to  be  implemented  in  pipelined  form.  The  major  existing  Look-Ahead  forms  are  briefly  outlined  first. 
2.1.  Clustered  Look-Ahead  (CLA)  M- stage  CLA  pipelining  of  this  filter  would  have  the  following  form:  [3 


4,  9,  13], 


ttM r  \  _  1  +  fti £  1  +  b2z  2  + hh+M-iz  (L+M 

C  {Z)  ~  1  +  aMz~M  +  aM+1z-(M+D  +  . . .  +  aM+v-xz-W+H-V 


(2) 


2.2.  Scattered  Look-Ahead  (SLA)  :  An  equivalent  M-stage  pipelining  of  the  same  N- th  order  recursive  filter 
can  be  obtained  by  [5,  12], 


M  _  1  +  M  1  +  b2z  2-fi - VbN(M-\)+LZ  (iV(M  1)+L) 


Bf(z)  = 


l  +  dM*  M  +  a>2MZ  2M  H - \-o>nmz 


-NM 


(3) 


Note  that  the  non-zero  denominator  coefficients  are  equally  ’scattered’. 

2.3.  Distributed  Look-Ahead  Pipelining  (DLA)  [8,  see  the  previous  Section  also]:  In  this  new  look-ahead 
scheme,  the  filter  transfer  function  is  transformed  to  have  the  form, 


rrM(  \  _ l  +  6:z  ]_+ - hhi+kL ,-N+LZ  JV+£) _  /4v 

D  1  +  aMz-M  +  aM+t^-^+b)  +  + - \-aM+kLz-(-M+kL'>  K 

where,  Jbi v  Jb2>  •  •  in  general,  can  be  arbitrary  integer  values  with  Jcl  =  A,  in  order  to  keep  the  total  number 
of  denominator  (a;)  coefficients  same  as  in  the  original  denominator  in  (1).  It  is  easy  to  show  that  the  two 
existing  look-ahead  schemes  defined  in  equations  (2)  and  (3)  are  special  cases  of  this  general  M-stage  look-ahead 
representation.  Specifically,  in  case  of  CLA,  ki  =  i  and  for  SLA,  ki  =  M«,  with  k^  =  (N  —  1)M. 

2.4.  General  Distributed  Look-Ahead  Representation  of  Recursive  Filters  :  In  general,  any  of  the 
above  M-stage  Look-Ahead  Pipelined  transfer  functions  can  be  obtained  from: 


_ l  +  friz-1  +  ■>+bQz-Q _ 

1  -I-  aMz~u  +  aM+k1z~(M+k^  H- - b  aM+kLz-^M+k^ 


A 


Bm(z) 

A(z) 

h( 0)  +  Ml)*"1  +  ...  +  h(K-  l)z-(Jf_1)  +  •  •  • 


(5) 

(ба) 

(бб) 


where,  appropriate  choice  of  Q  and  a  set  of  fci,  }  &L  would  lead  to  any  of  the  desired  Look-Ahead  forms 

in  (2)-(4).  Note  that  the  DLA  representations  in  (4)  and  (5)  differ  only  in  the  choice  of  the  numerator  order  Q 
which  need  not  be  restricted  for  the  approximation  algorithm.  In  fact,  by  choosing  Q  lower  than  those  required 
by  (2)-(4),  the  total  number  of  coefficients  for  the  Look-Ahead  representation  can  be  reduced,  if  desired. 

3.  Proposed  Method  for  Optimal  Estimation  of  Coefficients  of  Look-Ahead  Pipelined  Recursive 
Filters 
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The  CLA,  SLA  and  DLA  approaches  to  pipelining  are  two-step  design  processes  which  enforces  "exact 
equality”  to  an  already  existing  H(z).  Hence,  even  though  the  filter  orders  are  significantly  higher  after  any  of 
the  Look-Ahead  transformations  are  applied,  the  characteristics  of  the  filters  do  not  change.  Clearly,  the  original 
"lower  order”  H (z)  must  have  been  obtained  via  some  kind  of  approximation  approach  to  match  certain  desired 
time-domain  or  frequency-domain  specifications.  It  is  well  known  that  a  superior  fit  can  be  achieved  with  higher 
filter  orders.  However,  in  case  of  the  existing  LA  schemes  no  attempt  is  made  to  exploit  the  extra  degrees  of 
freedom  of  the  higher  order  Look-Ahead  realizations  in  order  to  achieve  superior  match  than  the  original  H(z). 
In  this  Section,  we  propose  to  use  an  optimal  least  squares  approach  [7]  to  design  the  look-ahead  recursive  filters 
directly  in  a  single  step.  The  goal  is  to  achieve  superior  match  to  the  original  specs  with  lower  filter  orders  than 
otherwise  would  be  needed  with  the  existing  two-step  procedures. 

Let, 

hd  =  [hd(0)  Ml)  •••  hd(N-l)]T  (7) 

denote  the  desired  impulse  response  (IR)  of  the  pipelined  (or  un-pipelined)  filter.  Our  goal  is  to  estimate  the  a,- 
and  b{  coefficients  in  (2),  (3)  or  (4)  so  as  to  match  this  desired  IR  specification.  Since  the  general  DLA  expression 
in  equation  (5)  includes  all  the  possible  LA  representations,  we  will  outline  only  steps  to  determine  the  coefficients 
of  the  general  M- stage  DLA  representation  in  (5). 

Stacking  the  first  N  significant  IR  samples  of  HM(z)  in  (6),  define, 

h=[A(0)  h(  1)  •••  h(N-l)f.  (8) 


The  problem  of  estimating  the  LA  coefficients  to  match  a  given  hd  can  be  stated  as  follows, 


N—  1  D/ ~\  I  ^ 

m*na  b  ||e||2  A  mm  *<»(*)  A(  \  {*(*)) 

=  a,b  *•— '  A[  Z  ) 

*=0  L  v  7  J 

(9) 

where, <5(1; )  =  {  q’  J  J  J, 

(10) 

e  A  —  h, 

(10a) 

a  A  [1  aM  ■■■  ajtf+ifcJ^and 

(106) 

b  A  [1  ii  •••  6 q]t. 

(10c) 

Rewriting  (6)  as 


Bm(z)  =  Hm(z)Am(z) 


and  equating  the  coefficients  of  equal  powers  of  2  1  on  both  sides  of  this  equation, 
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...  Mi)  • 

0 

-HQ)  • 
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.  h(N) 

••  h(N  -  M  -kL). 

The  matrix  J  in  (11)  is  necessary  to  constrain  some  of  the  coefficients  in  the  denominator  to  be  zero,  and  it  is 
formed  as  follows:  Starting  with  the  identity  matrix  of  size  (M  +  ki  +  1),  remove  all  columns  corresponding  to  the 
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indices  of  those  coefficients  of  the  denominator  which  are  zero.  The  remaining  matrix,  of  size  1)  x  (L+ 2), 

becomes  J.  From  the  bottom  partition  of  (11),  i.e.} 

H2Ja  =  0, 

it  can  be  shown  [7]  that  the  minimization  problem  stated  in  (9)  is  theoretically  equivalent  to  first  solving  for  the 
denominator  in 

rnin  aTJTH^(ATA)-1H2Ja  (13a) 

and  then  estimating  the  numerator  from  the  top  portion  of  (11),  i.e.} 

b  =  Hi  Ja.  (136) 

Note  that  the  matrix  A  is  a  banded  Toeplitz  convolution  matrix  and  can  be  defined  similar  to  H2,  as  given  in  [7]. 
The  design  problem  has  thus  been  decoupled  into  two  separate  sub-problems  of  reduced  computational  complexity. 

Note  that  the  denominator  criterion  in  (13a)  has  a  weighted-quadratic  structure  where  the  weight  matrix  in  the 
middle  itself  depends  on  the  unknown  coefficients.  For  estimating  a,  an  iterative  algorithm  has  been  presented 
in  [7],  where  the  estimates  at  the  previous  iteration  is  used  to  form  the  middle  matrix  (ATA)“1.  An  appropri¬ 
ate  modification  of  that  algorithm  can  be  used  here  to  minimize  (13a)  to  obtain  the  optimal  estimates  of  the 
denominator  a,  from  which  the  numerator  b  can  be  computed  using  (13b). 

4.  Simulations  on  One- Step  Approximation 
4.2.  Simulation  1 

In  this  case,  a  lowpass  example  has  been  considered.  In  all  figures,  the  solid  line  denotes  the  desired  response 
and  the  dashed  line  corresponds  to  the  response  using  the  proposed  approximation  approach.  Figure  1(a)  shows 
the  response  of  the  un-pipelined  filter  approximation  with  numerator  and  denominator  orders  =  3.  Note  this 
response  would  remain  identical  if  the  SLA  filter  is  obtained  from  it  using  Parhi’s  approach  [5].  The  error  is 
—9 .9dB.  Figure  1(b)  shows  the  response  of  the  CLA  filter  approximation  with  Q  =  6,M  —  4&i  =  1  and  k2  =  2 
and  has  an  error  of  —19.5 dB.  Figure  1(d)  shows  the  response  of  the  DLA  filter  with  Q  =  6,  M  =  4,  ki  —  2  and 
k2  —  4  with  an  error  of  —32.5 dB.  Finally,  Figure  1(c)  shows  the  response  of  the  SLA  filter  designed  directly  by  the 
OM-LA.  For  Figure  1(c)  —80.7 dB.  It  is  evident  from  the  error  values  and  a  comparison  of  Figures  1(a)  and  1(c) 
that  the  SLA  filter  designed  directly  by  OM-LA  is  much  superior  to  that  of  Parhi  [5].  The  respective  pole-zero 
plots  of  the  filters  are  shown  alongside.  Note  that  the  OM-LA  does  not  produce  cancelling  poles  and  zeros. 

4.2.  Simulation  2 

A  notch  filter  example  has  been  considered  for  this  example.  Figure  2(a)  shows  the  response  of  the  un-pipelined 
filter  with  numerator  order,  L  =  8  and  denominator  order,  N  =  10.  The  responses  with  the  CLA  ,  SLA  and 
DLA  approximations  are  shown  in  Figures  2(b),  (c)  and  (d)  respectively.  Note  the  filter  order  and  hardware 
requirements  for  the  SLA  filter  would  be  extremely  high  even  for  the  pipelining  stage  of  M  =  3.  Table  1  shows  a 
comparison  of  the  hardware  used  in  the  three  cases.  From  Table  1  and  the  responses  in  Figure  2,  it  is  apparent 
that  stable  CLA  or  DLA  approximations  can  be  achieved  with  excellent  match  and  much  reduced  hardware 
requirements  than  SLA. 
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figure  1:  Lowpass  Filter:  (a)  Response  of  an 
c)  Response  of  the  SLA  filter  obtained  directly  through  OM-LA  (d)  Response  of  the  DLA  filter 

Legend:-  -  desired,  -  OM-LA 
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Figure  2:  Notch  Filter:  (a)  Response  of  an  un-pipelined  filter  (b)  Response  of  the  CLA  filter 
(c)  Response  of  the  SLA  filter  obtained  directly  through  OM-LA  (d)  Response  of  the  DLA  filter, 
Legend:-  -  desired,  -  OM-LA 
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Section  -  3.9  PIPELINED  LOOK-AHEAD  IMPLEMENTATION  OF  A  CLASS  OF  2-D  HR  FILTERS 
SUMMARY 

In  the  previous  section  we  have  presented  a  new  scheme  (referred  to  as  distributed  look-ahead)  which  is 
a  compromise  between  the  two  existing  look-ahead  approaches  for  high  speed  implementation  of  1-D  Recursive 
Digital  filters.  To  date  neither  the  Scattered  Look-ahead  nor  the  Distributed  scheme  has  so  far  been  utilized  for  2- 
D  HR  filter  implementation,  primarily  because  the  1-D  stability  properties  of  these  LA  schemes  do  not  necessarily 
translate  to  general  2-D  HR  filters.  The  primary  focus  of  this  paper  is  to  demonstrate  that  for  a  special  but  very 
important  class  of  2-D  IIR  filters,  namely  for  Denominator  Separable  configurations,  the  benefits  of  these  stable 
look-ahead  schemes  can  indeed  be  taken  advantage  of.  The  efficiency  of  the  proposed  implementations  and  the 
reductions  in  multiplication  and  delays  are  demonstrated  with  some  examples. 

I.  Introduction  : 

Two-dimensional  (2-D)  IIR  filters  have  many  practical  applications,  such  as  in  radar,  digital  image  processing, 
remote  sensing,  etc.  Processing  time  and  throughput  delay  are  two  of  the  major  problems  for  implementing 
2-D  digital  IIR  filters.  Look-Ahead  (LA)  pipelining  has  been  found  to  be  highly  effective  for  attaining  high 
sampling  rate  and  high  computation  speed  for  low-cost  VLSI  implementation  of  recursive  digital  filters  [1-6].  In 
particular,  the  Clustered  Look-Ahead  (CLA)  scheme  has  been  utilized  for  implementing  both  1-D  and  2-D  IIR 
filters  [2].  However,  it  is  known  that  even  for  the  1-D  case  CLA  can  not  assure  stability  [2].  In  order  to  avoid 
the  stability  problems  of  CLA,  several  other  LA  schemes  have  been  proposed,  namely,  Scattered  Look-Ahead 
(SLA)  [2],  Minimum  Augmentation  CLA  (MACLA)  [3]  and  Distributed  Look-Ahead  (DLA)  [9].  To  the  best 
of  our  knowledge  these  later  schemes  have  not  so  far  been  utilized  for  2-D  IIR  filter  implementation,  primarily 
because  the  1-D  stability  properties  of  these  LA  schemes  do  no  necessarily  translate  to  general  2-D  IIR  filters. 
The  primary  focus  of  this  paper  is  to  demonstrate  that  for  a  special  but  very  important  class  of  2-D  IIR  filters, 
namely  for  Denominator  Separable  configurations,  the  benefits  of  these  stable  look-ahead  schemes  can  indeed  be 
taken  advantage  of. 

Separable-Denominator  2-D  IIR  filters  have  considerable  practical  applications.  Firstly,  many  commonly  used 
2-D  filters  such  as,  Gaussian,  Laplacian-Gaussian,  Lowpass,  Bandpass,  are  known  to  possess  symmetric  spatial 
response  and  hence,  many  of  these  filters  inherently  conform  to  denominator-separable  transfer  functions  [7,  8, 
11-13].  Second,  a  general  2-D  filter  can  be  approximated  by  a  denominator-separable  filter  [12,  13].  Thirdly,  the 
design  of  2-D  separable-denominator  filters  are  much  easier  and  each  of  the  1-D  denominators  can  be  implemented 
using  highly  modular  structures  [11].  But  most  importantly,  the  stability  tests  for  these  filters  are  simpler  and 
identical  to  those  for  1-D  filters. 

Direct  form  realizations  of  2-D  denominator  separable  IIR  filters  have  been  attempted  [11,  13],  but  have 
certain  speed  disadvantages.  Block  filtering  techniques  [9]  with  a  combination  of  scattered  look-ahead  and  de¬ 
composition  based  pipelining  [2]  can  be  used  as  an  approach  for  implementation  of  2-D  denominator  separable 
filters;  this  approach  being  carried  out  on  each  of  the  two  separated  domains  of  the  denominator  of  the  2-D 
denominator  separable  filter.  However  the  state  update  in  the  Block  filters  is  based  on  clustered  look-ahead  [9] 
approach  which  does  not  necessarily  guarantee  stability  and  at  the  same  time  block  structures  are  highly  complex. 

In  this  paper,  we  show  that,  utilizing  the  SLA  and  DLA  pipelining  techniques,  high-speed  modular  imple¬ 
mentation  of  separable-denominator  stable  2-D  IIR  filters  is  indeed  feasible.  It  may  be  noted  that  the  various 
stable  LA  schemes  recast  the  way  the  the  output  is  generated  by  appropriate  placement  of  the  1-D  poles.  The 
numerator  does  not  play  any  role  in  stability  considerations.  Hence,  if  the  original  1-D  factors  are  stable  to  begin 
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with,  application  of  any  of  the  stable  LA  schemes  to  individual  1-D  factors  would  also  maintain  stability  for  the 
overall  2-D  filter.  Block  Processing  for  further  speed-up  is  also  feasible  for  the  proposed  architectures. 


II.  Look-Ahead  pipelining  for  2-D  Denominator- Separable  HR  filters 


A  denominator  separable  2-D  HR  filter  transfer  function  is  given  by 


H(zuz2) 


_  A(zlfz2) 

D(zuz2)  ~  B(zi)C(z2) 


Em  1  \—^M2 

i=oLj=Ofltj,i?l 


(1) 


II.  1.  Clustered  Look-Ahead  Pipelining  [1,  2,  6]: 


M-stage  pipelining  of  an  (JVi,  N2)-th.  order  separable-denominator  2-D  HR  filter  can  be  represented  as, 
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where,  5s  denote  the  coefficients  of  the  extra  numerator  polynomial  introduced  due  to  the  pipelining  of  the 
denominator  [2,  6].  The  multiplication  complexity  is  (2Ai  +  M)(2A2  +  M)  and  the  latch  complexity  is  linear  in  M. 
The  extra  delay  in  producing  output  is  M  for  each  domain.  It  may  be  noted  that  this  scheme  may  suffer  from  the 
same  stability  problems  of  its  1-D  counterparts  [2].  A  Clustered  approach  that  guarantees  stability  with  minimum 
augmentation  of  order  (MACLA)  may  be  utilized  [3].  However,  finding  the  coefficients  for  MACLA  appears  to 
be  somewhat  cumbersome.  Hence,  in  our  examples  we  will  use  SLA  and  DLA  which  are  briefly  outlined  next. 


II. 2.  Scattered  Look-Ahead  Pipelining  [2]: 

An  equivalent  M-stage  pipelining  of  the  same  (N\>  N2)-th  order  recursive  filter  can  be  obtained  by, 
_  A(zirz2)(l  4-  di>0zf 1  +  dp,!^1  -f  diyiz^lz2l  4 - h  dN^M-i),N2(M-i)Zl  Nli^M  ^ z2  N^M 


H(zuz2)  = 
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(3) 


The  total  multiplication  complexity  is  (AiM+M-i-l)(7V2M+M-|-l)  and  the  latch  complexity  is  square  in  M  in  each 
domain.  The  extra  delay  in  producing  output  is  N\(M— 1)4* N2(M—l).  However,  if  M  is  a  power  of  2,  then  using  a 
decomposition  technique  [2], the  total  multiplications  can  be  reduced  to  (2N\+Ni  log2  M4-l)(2iVr2  4-Ar2  log2  M+l). 

II. 3.  Distributed  Look-Ahead  Pipelining  [9]  : 

For  this  recently  proposed  look-ahead  scheme,  the  filter  transfer  function  (2a)  is  transformed  to  the  form 
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(4) 

where,  fci,  Ar2,  •  •  •  are  integers.  It  is  easy  to  show  that  the  look-ahead  schemes  in  (2)  and  (3)  are  special  cases  of  this 
general  M-stage  look-ahead  approach.  In  [9],  the  stability  conditions  for  a  few  low-M  cases  have  been  presented. 

The  examples  below  will  demonstrate  that,  when  compared  with  SLA,  this  new  scheme  can  produce  stable 
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implementations  with  lower  multiplication  and  latch  complexities  and  reduced  output  delay.  In  our  examples, 
ki  =  N\  —  N2  =  N  will  be  used.  Clearly,  if  (M  -f  &/,)  <  NM ,  this  scheme  would  provide  considerable  savings 
over  the  scattered  approach. 


III.  Examples  of  Look-Ahead  Pipelined  Implementation  of  2-D  Denominator- Separable  HR  Filters: 

In  this  Section,  we  show  that  for  2-D  Denominator-Separable  HR  filters,  any  of  the  pipelining  schemes  dis¬ 
cussed  above  can  be  easily  adopted  for  high-speed  implementation.  Specifically,  denominator-separable  designs 
of  many  commonly  occurring  2-D  filters  such  as,  Gaussian,  Laplacian,  Lowpass  and  Bandpass  provide  excellent 
match  [7].  We  provide  two  examples  using  Gaussian  and  Laplacian  filters  to  show  that,  with  separable  denomina¬ 
tors  in  two  domains,  the  recursive  sections  can  be  easily  transformed  to  pipelinable  forms  using  any  of  the  forms 
in  (2),  (3)  or  (4). 


III.l.  Example  1  :  Gaussian  Filter  Implementation 


Consider  a  4-stage  (M  =  4)  implementation  of  a  (4,4)  order  2-D  Gaussian 


IIR  filter  with  the  following 


coefficients, 


A(zi,z2) 


-  0.00936980949545 
-0.0012672373559 
0.00825275276913 
.  0.00489009121014 


-0.00126723735355 

0.000171389878746 

-0.00111615893404 

-0.00066136950242 


0.00825275276870 

-0.0011161589361 

0.00726887011928 

0.00430710077865 


0.00489009121304- 

-0.0006613695040 

0.00430710078143 

0.00255213215053. 


(5) 


B(z  i)  =  l-2.2195z1_1  +  2.0846zr2  -  0.9754z1_3  +  0.19065zr4  (6a) 

=  (1  -  0.96486zf 1  +  0.4557zf  2)(1  -  1.2546Z;-1  +  0.41834zr2)  (66) 

A  B1(z1)B2(z1)  (6c) 


In  polar  form  (z*  =  rje±jS’),  B(z i)  has  four  roots  (poles)  with  radii,  r\  =  0.6758  and  r2  =  0.6467  and  angles, 
6\  =  ±44.38  and  02  —  ±14.18,  respectively.  Because  of  symmetry,  C(z2)  has  identical  coefficients  as  B(zi). 
Hence,  the  poles  of  C(z2)  are  identical  to  those  of  B{z\).  Each  second  order  of  B(zy )  and  C(z2)  are  pipelined 
separately.  Incidentally,  the  clustered  approach  produced  unstable  filter  in  this  case  (refer  to  the  pole-zero  plot  in 
Fig.  1).  Hence,  the  clustered  implementation  given  in  [6]  will  not  be  suitable  for  pipelining  this  particular  filter. 
The  scattered  approach  can  certainly  be  used,  but  the  coefficients  in  (6)  also  satisfies  the  stability  conditions 
given  in  [9].  Hence,  the  recently  proposed  distributed  look-ahead  scheme  [9]  would  provide  stable  filters  with 
considerable  hardware  savings.  Next  we  provide  the  coefficients  for  equivalent  pipelined  filter  implementations 
using  both  SLA  and  DLA  schemes. 


Scattered  Look-Ahead  Implementation  : 


It  can  be  shown  that  4-stage  pipelining  of  the  second-order  factors  in  (6b)  have  the 


following  forms, 


1  _  (1  +  0.0195zj~2  +  0.2077zf4)(l  +  0.9649zf 1  +  0.4557zf 2) 

Bi(zi)  “  1  +  0.4150z,-4  +  0.0431zr8 

1  _  (l±0.7374zf2±0.1750zr4)(l±  1.2546zf1±0.4183zf2) 

J92(zi)  "  1  —  0.1938zj"4  +  0.0306zf 8 


(7a) 

(76) 


Pipelining  of  Ci(z2)  and  C2(z2)  would  also  produce  identical  coefficients. 
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Distributed  Look-Ahead  Implementation  : 


In  this  case,  4-stage  pipelining  of  the  second-order  factors  in  (6b)  would  have  the  following  forms, 


1  _  (1  + 0.9649^ +  0.4557*r2)(l  +  0.0195zr2) 

BiC*i)  1  +  0.2073zf  4  ±  0.0040zj"6  (8a) 

1  _  (1  +  1.2546zf1  +  0.4183zf 2)(1  +  0.7374zf2) 

b2(zi)  l-o.sesszf^o.miz,-6  ' 

Because  of  symmetry,  Ci(z2)  and  C2(z2)  will  also  have  identical  coefficients. 

It  may  be  emphasized  here  that  both  SLA  and  DLA  implementations  in  (7)  and  (8),  respectively,  produce 
stable  high  speed  structures.  However,  comparing  the  denominators  as  well  as  the  numerator  factors  in  (7)  and 
(8),  it  easy  to  see  that  the  recently  proposed  DLA  scheme  provides  considerable  savings  in  multiplication  and 
latch  complexities  and  reduced  delay  in  output  generation.  The  pole  locations  with  4-stage  lookahead  for  this 
example  are  shown  in  Fig.  2.  In  Figs.  3  and  4,  the  signal  flow  diagrams  are  given  for  a  pair  of  2nd  order  blocks 
in  two  domains  for  4-stage  scattered  and  distributed  look-ahead  pipelining,  respectively.  Comparing  the  number 
of  delays  and  the  multipliers  in  Figs.  3  and  4  also  it  is  obvious  that  DLA  can  offer  reduced  complexity  than  SLA. 


III.2.  Example  2  :  Laplacian  Filter  Implementation 

In  this  case,  a  (4,4)  order  2-D  Laplacian  HR  filter  with  the  following  coefficients  are  considered, 


A(zi,z2) 


’ -0.00234823839532  -0.00391545411710  -0.02106505604166 
-0.00373345042666  0.00476642973501  -0.01976926214265 

-0.02194869729514  -0.02059991460608  -0.04640872036691 
0.00165842866327  0.01338997088113  0.1370918579269 

.-0.02470285791456  -0.02800618927861  -0.09758570689236 


-0.00234522754996  -0.0230375942386 


0.01122239739174 

0.14204249387362 

0.1639249127551 

0.11045024575180 


-0.0264271437586 

-0.1034676446472 

0.09669575185083 

-0.1501644414553 

(9) 


B(z i)  =  1  -  1.64180ZJ-1  +  1.50063zj"2  -  0.80133zx-3  +  0.21866zf4  (10a) 

=  (1  -  0.4852zf 1  +  0.5146zf 2)(1  -  l.lSeezf1  +  0.4249zf2)  (106) 

A  B1(z1)B2(z1)  (10c) 

The  radii  and  the  angles  of  the  roots  of  B(zl)  are,  rx  =  0.71736,  r2  =  0.65185  and  0X  =  ±70.23  and  02  =  ±27.48, 

respectively.  The  poles  of  C(z2)  also  have  the  same  values.  Similar  to  Example  1,  each  second  order  of  B(zi)  and 

C(z2)  are  pipelined  separately.  The  clustered  approach  of  [6]  again  produced  unstable  filter  in  this  case  (due  to 
B2(zi)).  However,  in  this  case  also,  the  coefficients  in  (10b)  satisfied  the  stability  conditions  of  DLA  [9].  Hence, 
both  the  scattered  and  distributed  look-ahead  pipelining  methods  can  provide  stable  filters  with  DLA  providing 
more  hardware  savings  than  SLA.  The  coefficients  for  scattered  and  distributed  pipelining  implementations  for 
this  example  are  given  next. 


Scattered  Look-Ahead  Implementation  : 


It  can  be  shown  that  4-stage  pipelining  of  the  second-order  factors  in  (10b)  have  the  following  forms, 

1  _  (1  -  0.7938zf 2  +  0.2648z1~4)(1  ±  0.4852z1~1  +  0,5146zx-2) 

Bi(zi)  ~  1  -0.1005zr4±0.0701zr8 

1  _  (1  +  0.4879zf 2  +  0.1805zf 4)(1  +  U566zf^  +  0.4249zf2) 

B2(zi)  l±0.1231z1_4±  0.0326zf8 
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Ci(z2)  and  C2(z2 )  would  also  have  identical  forms. 

Distributed  Look-Ahead  Implementation  : 

in  <10b)  of  ,he  Up,acian  mt"  "hen  pipeiiMd  * 


i 


1 


-  =  (1  +  Q-4852^-1  +  0.5146^r2Kl  +  0.7938z,~2) 


1  -  0.3653ZJ  4  -  0.2102zf 6 

(1  +  1.1566Z;-1  +  0.4249*r2)(l  +  0.4879z,~2) 
1  -  0.0575zf4  +  0.0881z~6 


(12a) 

(126) 


*•  schemes  fot  *, 
tions.  Equations  (11)  and  (12)  furth  b  tv.  T  ^  °WS  ^  b°th  SLA  ^  DLA  provide  stable  implementa- 
and  reduced  ^  T  /  ^  "  [9]  ^  P™ide  red-d  complexity 

will  be  included  in  ^  k  ?  ^-denominator  2-D  HR  filters.  More  examp, I 

paper  mcorporatmg  block  processing  [10]  for  further  improvement  in  throughput. 
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